Apart from not getting a House with a bigger Garden which i would presume a lot of people would say.
Mine would be building a pond that takes up most of my postage size Garden, because i was into fish long before i was into Plants, although it was the pond that got me into plants and the exotic scene
The second is not being able to get rid of a plant, and now the Basjoo's are taking over
Loads !! Nannorrhops, Trithrinax campestris, Agave montana to name but 3 with wildy exaggerated claims of hardiness/suitability to the British climate.
My Yucca gloriosa took over a space about 11' square and nearly 6' high, until 2010/11 when it also looked very tatty so I cut it down to the ground so it could start again, which it has and looks good but will keep it in check this time.
Ground elder
No regrets for me. The plants I have lost have been an education towards helping me keep plants I still have. Nothing is beyond control if I really want to make a change and as for what I'd wish on my worst enemy, well it wouldn't be a plant.
Main regrets are Bamboo and Ivy also would not wish them on my worst enemy.
Another regret is not evaluating the size palms grow. The big Chamaerops planted at 1ft, now completely shades the patio for much of the day, and shades the conservatory in winter
spending good money on varieties of palm seeds. conclusion: most fail or take too long.
buying many small palms instead of a few large ones: false economy by the time you feed, protect, repot year after year you are just as well to have bought big healthy ones with more impact in the garden
exceptions: washingtonia seeds are free to collect abroad easy to germinate and fast growing. well worth planting out a few small trachies if they are cheap.
Not doing the some things early enough / immediately when we moved in here.
The main one was not planting wind-break trees around the perimeter of my property (believing it was too narrow to accommodate them). Hedges too, plant them as soon as you arrive at the property. If you get there and the hedges are Leylandii, or something you don't like, pull them out and plant what you do want; they are going to be 5 years before they are anything significant.
Get any trees in, even if they turn out to be wrong. Easier to remove them, or attempt transplant, rather than wait for new ones to grow when realisation dawns that trees were needed.
Similarly, worth planting things in a nursery bed to gain a year, or more, of growth and take the risk of transplanting. I have done that with hedge plants, and some others, and am very glad that I did.
I wouldn't wish horsetail on anyone. Bindweed and Ground Elder respond to Roundup/Glyphosate. Horsetail is harder to eradicate.
Off topic: Many years ago I asked my wife's Grandmother (then in her 90's) what was the best thing she had done in her life and she said "Going on a cruise to Acapulco" !! I then asked her "Is there anything you regret not having done?" ... "Yes" she said "Not going to Acapulco a second time" !!
and the exotic scene 
also would not wish them on my worst enemy. 
Kris,
Virgina creeper/Boston Ivy---Good fence camouflage but likes to do the same to anything that stands in its way. 
