yes mate, basically as it's a "HD" wallet you can recover all private keys using the 24 (BIP39) word phrase:
But you'd need to move all funds (all coins) out elsewhere; wipe the Ledger, set a new BIP39 key to be safe.
I'd wait until Ledger supports Litecoin cash personally, then it will just show.