Let me preface this post by saying I have no problem with Vancian magic. While I've had players who didn't care for it (read: hated it), I've never seen where all the bile comes from. I rather like the idea of spells as a resource and I typically try to play Magic Users when I play D&D so I've obviously never had a problem with it.
That being said, sometimes I want a little more out of magic. I want foul acts performed in the dark of the Moon. I want dust created from the powdered bones of nine mummies from the steaming Jungles of Mu. I want alchemical tar that when rubbed on the eyes of a corpse causes the deceased to return to his or her body and whisper secrets from beyond the grave. I want salt circles to hold demons so that the sorcerer can force their demonic tutors to reveal ancient sorceries. I want hooded figures chanting around a stone idol. You know, that sort of thing.
This has some precedence in D&D. Most notably in Dave Arneson's rules that would lead to D&D wizards had to use strange formulas to concoct various "spells." Or something like that. I don't have the First Fantasy Campaign to reference, but Rob Conley hints at it a bit in his discussion of that book and Dragons At Dawn contains a system supposedly based on Arneson's rules. Granted Dragons At Dawn might have gotten it wrong, but it's neat enough that I hope Daniel Hugh Boggs is correct.
While not entirely the same as ritual magic, this system does appeal to me. More recent examples, assuming the Arneson anecdote is correct, include Geoffery McKinney's infamous work Supplement V: Carcosa and Rob Conley's Supplement VI: The Majestic Wilderlands. These two have very different takes on how ritual magic should work. Conley's is largely based on 4e's ritual system. It couples Vancian magic with the ability to pay gold and take several minutes to cast any spell the Magic-User knows as a ritual. I'm not terribly keen on this, but it's a serviceable solution. I've thought about adapting it to Nightwick Abbey, but I'm unlikely to in the near future, but it is a possibility.
I think McKinney's model, despite being squicky, is a tad closer to what I want. Now, I am not going to represent the rituals in the kind of lurid detail he provides, but the idea that individual spells require rituals and can never be cast instantaneously interests me. It could be combined with an otherwise Vancian system. Perhaps summoning a monster takes time and tools, but producing a fireball doesn't. This would be in line with the Roger Corman film The Raven, which I have cited as a source of inspiration before. Another way of looking at this model could be that any spell over level 5 is a ritual of some form.
They could also be coupled with Arneson's model. In this version, a wizard would ready the powders and ichors required for whatever challenges he or she believes he will face in a dungeon or other discrete adventuring local. However, while at home he or she might take days to complete some sort of ritual to summon a familiar or create a terrible storm.
Finally, the rituals could stand on their own as they do in Carcosa. Magic is not something cheap and reliable. It is strange and difficult. Magic takes time to practice and master and invoke.
I'm going to be considering these models as I develop my two settings further. While Nightwick Abbey and the Dark Country will still rely on Vancian magic, I would like for the various sacrifices made to terrible demons to actually do something. The Underworld is still up in the air. It might use Vancian magic, or I might develop a ritual magic system for it.
Sorry to those who thought I would have rules for Ritual Magic in this post. Perhaps I will once the ideas I have get more concretely formulated in my head.