OK, thanks for the tip TNhunter!
Initially, I was worried that my prime growing areas would get mowed down by deer, but I'm hoping that planting it in the middle of all of the vegetation will help hide and protect it, and so far so good with my older larger plants.
I know everybody does things different, and next year I might try a few small areas with better site preparation in a woodsgrown type of approach (spraying weeds, amending soil, and/or tilling), but I just like the idea of keeping it as wild as possible.
