The trouble with rating media

You may have noticed that I don't really include ratings on book. I tend to shy away from the practice as I often ffind it hard to know how to rate things like movies, books and TV shows.  You know things that will be compiled and make recommendations for future suggestions.  I find it hard to know where the break points should be.  First I should say I am very stingy on giving out 5 stars, as I think that only perfection deserves 5 stars.  Gatsby and Breaking Bad fit this category, but perfection is hard to reach. 

From there you go to 4 stars.  This is where I tend to put things I really like and enjoyed and that are well done.  But even a 4 star rating can be hard for me to give out, because it has to be near perfection.

1 and 2 stars are a bit easier, actually, I wish there was a zero star for things I just hated.  But these are things I really don't like and it is easy for me to know when it just isn't for me.

And that leaves the 3 star ratings.  These are the hardest for me because there are 2 types of books, movies and TV shows that I partake in.  And a 3 star classic novel would be vastly different than a 3 star YA book.  Not to mention how do I reckon a bad story by Fitzgerald or Asten to a good fluff book.  When I rate Fitzgerald or Austen it is usually to their other work, but just because I liked it less than something else they have written dosen't mean it should resided in the realm of Divergent.  Divergent is a good YA novel.  I didn't hate it, so I wouldn't give it a 1 or 2, it is a 3 star book for me.  But when I place it next to A Farewell to Arms by Hemingway, it just seems wrong that they should even be rated the same.

I need  2 prong rating system. One for more serious works and one for fluff.  Does anyone else have this problem?  How do you deal with this?