When it comes to magic, what I do depends on the story that I’m telling and what I already know (and what I know I still need to find out) regarding whatever that type of magic requires. Those things have all evolved over the years as I figure out more and more about what the heck I am doing.
My first fantasies were the first two Lyra books: Shadow Magic and Daughter of Witches. I started making up that world when I was in my teens; I had no clue about all this writing stuff, but I had a feeling about what I needed in order to make the story work. And what it needed was a certain amount of magical structure, preferably something that rested on ground I was reasonably familiar with. So I based the magic and everything that went with it around the tried and true four elements: Earth, Air, Fire, and Water. I did very little specific research, but I wrote pages and pages of classifying-things-in-fours to see what kinds of things matched up (in my head, at least) with each element. Almost none of that got into the books in any obvious fashion, but it was all there in the back of my mind as I wrote.
My third book was Talking to Dragons, which I wrote without doing any prewriting at all. The magic was whatever showed up; it is perhaps unsurprising that what showed up eventually proved to be based on fairy tales, which I had been devouring in great quantities since before I was old enough to read (my parents were very good about reading to their pre-literate children). After reading enormous quantities of myths, legends, and fairy tales all my life (that section of my library is larger than anything but the SF/F section), I didn’t need to do prewriting or planning or even much in the way of thinking about how the magic worked. It was already sunk deep into the compost in my backbrain. I knew what felt right and what didn’t, even when I was making up things like fire witches and the quozzel.
The Seven Towers didn’t use elemental magic because it wasn’t set on Lyra, but the magic in it feels (to me, anyway) a lot like the magic of the Lyra books, partly because I did very little in the way of designing it besides reminding myself “This isn’t Lyra; it doesn’t go by fours” every so often. I was working on characterization and viewpoint in that book, so I more or less defaulted to what was familiar, with a few things thrown in to make sure it was different. If I’d gone on to write sequels, I would have had to get fairly creative about the logic behind some of those things, because they didn’t have enough thought put into them in the first place.
I went back to Lyra for The Harp of Imach Thyssel and Caught in Crystal, which was just as well – I had more technical writing stuff to work on, and doing it in a world where I wasn’t also trying to work out a new magic system made it easier to concentrate on. Around that time, I was also participating in the Liavek shared-world anthologies, for which we did make up a very specific system of magic in advance, with some very specific rules (which we and everyone else in the anthologies immediately set about bending and stretching to the breaking point). We wanted it to be relatively egalitarian – we wanted magic to be something that anyone could have access to, if they wanted it badly enough – but also dangerous enough that not everyone would try, and not something that would make doing things by magic a too-easy substitute for technology. I’m not sure which of us came up with the concept of birth luck (birth being something that everyone goes through, thus meeting the egalitarian test).
Then came Snow White and Rose Red, which was a semi-historical retelling of the Grimms fairy tale. That took a lot of research; it was set in Elizabethan England, in 1582-83, and two of the major characters (John Dee and Edward Kelly) were historical personages whose experimentation with magic and the occult was both well-known and well-documented. The Elizabethans also had very specific ideas about the magical properties of plants, and a great many equally specific ideas about fairies of different sorts, all of which had to be woven together to get the story I wanted. Dee and Kelly’s experiments were in the tradition of complex rituals, involving diagrams and special equipment, while the herbalism was verging on turning into modern medicine, and the fairies were subject to a host of their own rules with little apparent logic.
The next new venture was Sorcery and Cecelia, which, due to its genesis as a game played between writers, had no advanced planning of magic systems. Caroline and I threw in whatever seemed as if it fit and would make for an interesting letter, and justified it after the fact. I’m not sure it would have been possible to work that way if we had not both had a certain amount of familiarity with the period (England, 1811-1820) and with British history and British fairy tales and legends. The end result has a good many messy bits, where various real-world traditions fall over each other and seem contradictory, but a lot of real-life science was like that then, with many competing theories and no coherent overview, so we decided it worked. (But don’t ask either of us to explain what sort of Unified Field Theories of Magic the magical theoreticians came up with fifty or a hundred years after that time. We never made that up.)
Mairelon the Magician and Magician’s Ward were built on the research I’d done for Snow White and Rose Red, with a fair amount of additional reading about what people in post-Napoleonic-Wars England thought and believed in regard to the “occult sciences.” It was a bit before Mesmer and the séances of the Victorian era, and a bit after the Hellfire Club of the mid-Georgian period, but I read up on both and then adapted things freely to suit my stories.
From Snow White and Rose Red through the alternate-universe-Regency-England books, the magic systems were not made up out of whole cloth. Like the alternate history, they were more “what if all the things people believed about magic and how magic worked really were magic” than “how would having real magic change the way history would play out.” Building magic systems for those books, therefore, meant mainly reading up about what people in real life thought about magic, and then making use of the parts of it that fit my stories.
For the Frontier Magic books, I wasn’t trying to work with real-life parallels. I wanted a world where it looked as if there were several completely different and incompatible kinds of magic, but where the various schools of magic had simply developed different approaches to magic – rather as if classical orchestral musicians, jazz musicians, and the members of the high school band all insisted that what they were doing was completely different from what anyone else was doing, so that they thought a flute made to play in a high school band was fundamentally different and incompatible with a flute crafted to be played in an orchestra, and learning different kinds of music meant learning totally different skills, as different as spinning yarn and carving stone.
So I did a fair amount of up-front design of the magic system in order to come up with something that could have several different approaches that would look very different, but still be tapping into the same fundamental thing. Since I wanted my heroine to have to move toward teamwork (and I wanted doing so to be hard and non-intuitive for her), I made her part of the tradition that emphasized soloists and the difficulty and danger involved in magicians working together. The most collaborative magicians were both the most advanced and the farthest away (physically) from where she was; they didn’t actually show up in person until the third book (forcing her to figure out a lot of stuff on her own). I also put a fair amount of effort into the magical ecosystem, on the theory that if you have a magical apex predator like a dragon, there ought to be lots of other magical critters, in all niches of the natural world. (I never did get a chance to work in the magical bacteria, darn it.)
The current project also requires a considerable degree of careful advance planning for the magic system, as I want to ring some changes on an old fantasy trope. This means I have to have a good idea what the old standard version would look like, as well as thinking about where the cracks are that I can exploit later in my story. Outside of over-the-top parody, it seldom works to introduce a bunch of traditional vampires and then have the main character realize, right at the story climax when it’s needed, that they are all deathly allergic to lemonade and nobody has ever noticed this before.