|Having gone through this process for my mother's family, starting with very little information, here are my suggestions:
1) see if your family surname has a genealogy forum on the internet; just "google": "[Surname] famly forum" and the URL will come up; usually there is a "parent" of the forum (normally women) who have CDs from services, etc. and are family experts; most likely someone has already done your line or parts of it and you'll find cousins.
2) go to your local library anf find out what genealogical sources they have: database services to ancestry.com, etc. With ancestry.com you can view, not only the census, you ancestor is listed on, but also his CW service record--which doesnt' say much, unfortunately; also use the old census books, as sometimes mistakes are made in both.
3) if you have to back track from start--start with a gparent who was living in 1930 at the latest--this is the last year all the names of persons in a family were listed on the census individually (and 1850 the earliest)--go back as far as you can--identifying males by name you're looking for
4) find the county your ancestor was living in and find the county history done probably in the 1880s (for help go to http://www.nypl.org > CATNYP > keyword search); and some are being redone currently; look for other local information such as books on marriages. Use the nypl.org to look for family books, too.
5) go to http://www.findagrave.com -- look for grave site
By going back and forth between the Genealogy forums (also search related families), the census data, and county histories, I was able to quickly fill in most of my mother's line. However, sometimes, people just are not in the census (if they were not at home when the taker came, they did not get listed) or you have to go back very far (one ggggmother finally turned up in 1840 in NYC and was missing in the census book, but found on anceestry.com).
Also once you find the county your ancestor lived in you can write the county clerk for any records. The cost is about $2 a piece. The local library there might also have cemetery listings and other books.