While there are definitely fantastic guitarists that rarely (if ever) use their pinky, I would recommend putting time into it. We as guitarists have 4 digits we can utilize to express ourselves. The pinky is definitely the hardest to begin using since we use it so little in our every day lives...
What everyone here said about practicing slow and gradually going up is definitely the biggest piece. If you hit a plateau on a song or section in terms of speed, I suggest moving on to another song and using that difficult section from the original song as a warm-up drill. In my experience...