The first thing you need to do is buy a domain name. You can see if your chosen name is available from a site such as
http://buydomain.com/. Ideally you should chose a domain registrar and a host in the country of your target market. I notice you're in the States so I assume your main market will be there. I've only had personal experience of UK registrars and hosts so I'm unable to recommend any in the US, though no doubt someone one here will point you to a reputable company.
Once you've chosen your domain name, you need to decide what software to use. I would recommend one of the free Open Source packages such as Zen Cart. Zen will allow you to expand your store considerably at minimal cost. When you have chosen the S/W, you will need to chose a host. The host doesn't necessarily have to be the same company that the domain name was purchased from. For a store running Zen Cart, osCommerce or most of the other open source software you will need hosting with PHP & MySQL, virtually any host offering PHP & MySQL will be able to run Zen though, if in doubt, ask them first.
When you have all of the above, it's a simply matter of installing the S/W. This is usually a 10-15min job for a basic install. If this poses a problem, ask on here and either myself or someone else will either talk you through it, or even install it for you.
After doing all of the above, you will have a basic store that with a bit of basic configuration (straight forward using admin menu system) that is capable of selling your wares and taking payment by PayPal or cheque(check). I would then recommend installing a template that suites your store. You can choose from a number of free ones, though I'd recommend spending a small amount to have a professional template installed to give your eshop a more professional look. Have a look at Dezina's site for a wide choice, click her sig link on one of her posts on this forum.
If you later decide you need other features added to your site such as additional payment methods, these can be installed when needed.
The whole process isn't as daunting as it probably seems, once you've done it once, it'll all make a lot more sense.
To find out more about the Zen Cart S/W I mentioned above, have a look at
http://en.wikipedia.org/wiki/Zen_Cart
http://www.zen-cart.com/