It probably won't bother me either, but my reasoning is a bit more involved than yours. First of all, I'm not expecting anything from Social Security. I know what shape the system is in, and I've also alluded to the fact that it is likely the US may fall like the USSR before I'm old enough to collect. If the US goes, so does its Social Security obligations.
The rest of my reasoning has to do with what I pay, or rather don't pay, into the system. This year is the first year I'm filing as an independent contractor rather than an S-corporation (this is the only legal way I've found to avoid paying Social Security taxes). My goal is minimizing my taxes same as any other business, but after accounting for the fixed fees filing as an S-corp only costs less if I make about $10,000 or more. Therefore, until I consistently make over $10,000 per year I won't be filing as an S-corp any more. Let's say I don't have much luck, and make just short of that amount. I'll continue to file as an independent contractor for the next 25 years. I will have paid ~$30,000 into Social Security. Had I invested that myself instead and kept up with inflation, I would have had the equivalent of $30,000 25 years from now. Enough to last me two years if I'm lucky. Therefore, no big deal if I lose it because I don't collect Social Security. If I make significantly less than ~$10,000 then I will have paid even less into Social Security, and my losses are smaller. If I make consistently more, I will have filed as an S-corp and not paid anything. Therefore, I essentially capped my potential losses in Social Security to about $30,000 regardless of what happens with the system.
BTW, any reason you don't think you'll live past your 60s? Do people in your family have a history of dying fairly young? I'm just wondering because even my morbidly obese, sedentary grandmother died a few years ago at the age of 87. I tend to think any person taking reasonably good care of themselves these days can make it into at least their 90s, and probably well over 100. We got a Christmas card from one of my former neighbors this year who just turned 96. 60 or 65 is young by today's standards.