I must admit, I adore lovecraft and he has been a huge influence on me and my writing (I feel like I'm just parroting what others have said here. Hmmm....) While I do love his mythology I must admit his writing is what captures me. The way he uses words and phrases to express the terror and dread of the moment really inspires me. I've never read words that so properly capture the macabre so.
Most people won't find his stuff scary because the equate horror with shock and gore (A product of most horror movies these days). Things ambush people and then vivisect them in the most horrid ways and that's it. Lovecraft writes to express the mood of terror, the feeling of being mentally alone while your senses are being assaulted by things that don't belong in your worldview. Things whose very existence makes your entire life and indeed species an exercise in futility. To murder is one thing, to deconstruct your perception of reality and replace it with one of true horror is far more disturbing. That's at least the way I view it. The only thing I've ever seen do that is a videogame series called Silent Hill. Very good horror, surprising considering most videogame horror is B-movie quality.
Though ScottRadom, what do you dislike about his writing? I'm quite curious.