One of my favorite sites is the Army of Tennessee relic site by Nick Peirut. He always has odd, strange, scarce, and rare bullets - and his pictures are exceptional.
Another favorite site is Tennessee Lead by Ken Price. Again with excellent pictures. Most of his offerings are the more common bullets, but he surprises me frequently!!
I've acquired a lot of bullets from both places over the years. I maintain a "type" collection, meaning I collect only a single bullet of each pattern - CSA and USA - in the best condition I can afford. I'm frequently upgrading. As of today there are 828 bullets in the collection.
As for which ones are the most difficult to obtain is up to you. If you have deep pockets you can find them all. But that's a personal decision!!! I would suggest that you determine which ones you want and seek them all at once. You never know where/when you'll find the elusive pot of gold.
And just when you think you've found them all, another previously unknown variant will show up!!!!!!!!!!!!!!
Wes