There are two basic problems with controlling a mob of characters: juggling all their individual stories throughout a book, and juggling their conversations when all of them are in the same place and trying to talk at the same time.
The simplest solution to both problems is, obviously, to cut some of the characters, either out of a scene or out of the book entirely. Cutting characters out of the story can be hard to reconcile oneself to, but quite often it results in a tighter, more effective story, especially if the writer is not terribly good at juggling multiple backstories and conversations just yet. Surprisingly often, it results in a more interesting story, because cutting one or two characters forces the writer to consider alternate possibilities.
For example, one writer I knew started with an ensemble group of five main characters. Unfortunately, her world-building included a society that mandated early marriage and treating couples as a single unit (i.e., if you hire an engineer for a project, his/her spouse is part of the package). This instantly ballooned the cast to ten characters, and she hadn’t even gotten around to the bad guys yet.
First question was, did she really need all five of the main characters as she had originally conceived them, or could one of them be the spouse of one of the others? That worked, so she was down from ten characters to eight. Next question: could one of the characters be a widow/widower? That also worked, and provided an interesting opportunity to show how the society would deal with the inadvertently single. What about a gay couple? That didn’t work for the story, but she filed it as an interesting idea to explore at some other time. She was left with five main characters, but only two extra spouses, which was a vast improvement.
Cutting out characters can frequently be done by combining roles. For instance, in the current WIP, I started with a Head Minion, Evil Aunt, Evil Uncle, and Sort-of-Evil Cousin, among others. When I decided I needed to pare down the number of people, I looked at that and realized that there was no reason why the Head Minion couldn’t have gotten his position by marrying the Dark Lord’s sister, thus combining the roles of Head Minion and Evil Uncle in one person.
Cutting characters out of a scene is usually a matter of finding them something else to do while the scene is taking place. “I’d love to come to your meeting with the King of Somewhere and provide you with good advice, dear, but there’s a dragon attacking the town just to the west and I have to go sort it out.”
Sooner or later, however, one gets to a scene where everybody has to be present. Often, this is the Big Climax of the novel, and one can sometimes reduce the numbers present by killing off or disabling a couple of folks along the way. Still, there are probably far more people present than can have a reasonable conversation.
So think about a giant dinner party in real life. Four people at one table can all talk to each other at once, taking turns listening and speaking. Six people can do it sometimes, but if the center of the conversation is at one end of the table, the two people at the other end often start their own conversation. So with six people, you sometimes have one large conversation going, and sometimes two small ones. Anything more than six people is almost always split into two or three two-to-four-person individual conversations. It’s what people can hear and participate in over a distance.
This is surprisingly easy to adapt to writing. First, you figure out where everybody is in this giant scene. If it is a battle, and you have seven or more people involved, they will probably end up in different places, which means that instead of juggling seven-plus people in one scene, the writer can write three to seven individual scenes, each focused on just one or two characters and what is happening to them. Juggling then becomes a matter of cutting and pasting them into the most pleasing order.
If you have something like the grand finale of Mairelon the Magician, where I had sixteen characters in a room at once, all with different agendas and different ideas of what it was important to talk about…well, then you have to juggle the presentation of all the conversational bits. This means you have to figure out who is talking to whom (which starts with who is standing close enough to whom to be heard), and what sorts of overlapping conversations they are all likely to have.
You also have to consider your viewpoint. I like first-person and tight-third person, which give me a nice, clear way of organizing and juggling all the various two-to-three-person conversations going on: they are all filtered through what my viewpoint character is paying attention to at any given moment. If she’s talking to someone, all the other conversations are going to be like bits and pieces overheard in a restaurant; if she’s keeping quiet, she’ll pay the most attention to the people she thinks are the biggest threat or the most likely to have important or pertinent information or ideas.
I don’t tend to plan out my crowd scenes in any more detail than I do most of my other scenes, which is to say that if I am not sure who is where or how various moves will work, I sometimes do a sort of football-play-type diagram with letters representing characters and arrows showing how they’d move or respond to someone else’s movement. That works best for fight scenes, though there have occasionally been tea parties where I needed to know who was where and whether they had a clear route to an exit or a hiding place.
More usually, I have a clear idea of what everyone thinks they are about to do (which generally does not include the presence of any of the other characters – most of my crowd scenes are a surprise to the participants, these days), and then I just start with my main character and let her observe as people arrive. Every page or so, I stop to make sure that I haven’t lost track of anyone – even if the villain is in the middle of the traditional monolog, the viewpoint can see other people reacting in fear or loathing or exasperation, or sneaking off to hide behind the arras, or fiddling with their teacups. More usually, I have several two-to-five line excerpts from one two-to-three person conversation, then two-to-five-lines from a different one, as my viewpoint character tries to keep track of what everyone is doing and saying. If it gets too chaotic, I figure one of the characters will get fed up and yell “Silence!” loud enough to make everybody stop for a minute and start over.
It does take a fair amount of rewriting to make the conversation snippets in a mass crowd scene flow well. Sometimes, you can rearrange them; other times, one set has to come in a certain order, so all you can do is spread it out a bit and interpolate bits of other conversations, so that none of the other speakers gets lost. In the rewrite, the thing to keep an eye on is how long it has been since the last time each of the characters was mentioned. They don’t have to have actually said something; they can roll their eyes or scuff their feet or give some physical reaction to what someone else is saying. You just want to remind the readers who all is present, so that nobody will be shocked when someone who hasn’t been mentioned for ten or fifteen pages suddenly says something. (You also want to keep track of anybody who leaves; it is extremely awkward to have somebody deliver several crucial lines of dialog and then discover that they left the room five pages earlier.)