"Science fiction: is what we point at when we say it". -Damon Knight
It has always been difficult to find a nice more or less watertight definition for science fiction. Many have tried but none have succeeded. Me, I'm not even going to try, suffice it to say, like in the quote above I know science fiction or fantasy when I see it. Not all that surprising since I've been reading the stuff from when I was about six or seven.
Like many people, I started by reading what was available in the children's section of the local library. Fortunately this was reasonable stocked both with children's sf like the Dutch Euro 5 series as with classic books from writers like Isaac Asimov, Arthur C. Clarke, Robert A. Heinlein and A. E. Van Vogt. Later on I discovered writers like Vonnegut, Moorcock, Niven, Herbert and many others in the adult section of the library and when I started reading in English I really started finding out how big the science fiction genre is. Even then, I still naively thought I could read all science fiction and like it. Now I know better... Like any genre, 90 procent of science fiction is crud, but then again 90 procent of everything is crud.
Still, even without trying to read all the bad sf there are still too many good books being published for me to read them all; my backlog at the moment stands at around a 100+ books and growing... (There will always be books you haven't read yet: the question is whether you own them or somebody else does.)