opp, for some reason i read that it was the phasor not the envelope filter. have phasers on my mind latey.
anyways, it does say in the notes to use 3mm diffused red. the LED should be really bright, also, did you encase the LED and LDRs? if not that cold be part of the issue. you need to make sure that they are seals do that
ALL the light from the LED goes to the LDR, if not then it wont work properly and could make the effect sound distorted and messed up. you want the face of the LDR to completely face the LED. like they're kissing.
also, if you read all the discussion you'll find that this is very sensitive to high gain effects and pickups. so if you're using higher gain pickups that could be your issue causing the distortion, which is normal for this effect.
you may also want to just go over the board again and knife between the tracks, and check for poor solder joints and bad grounding.
good luck