More than once over the years, I’ve run into writers who complain about writing villains or who have trouble finding a “Big Bad” for their stories. Usually, they’ve somehow gotten the impression that every story has to have a villain of some sort, who has to be defeated utterly at the end of the tale. Whether this villain is an Evil Overlord, a space alien planning on conquering Earth, a supernatural demon (vampire, werewolf, evil force), or a sleazy lawyer or banker doesn’t matter much in their minds; they need a bad guy for their heroes to go up against, and they can’t come up with one that satisfies them.
In at least half these cases, the real problem has turned out to be that the author wanted to write some other kind of story, one that didn’t have a villain or Big Bad or monster-of-the-week. They already had a perfectly good plot, and adding a villain to it was a distraction, something that took away from the central point of the story, rather than adding to it. Naturally, under those circumstances they were having trouble finding a villain to fit their story – they didn’t have an empty niche in the plot for a villain to fit into.
The other half of the time, the author had a character who was supposed to fit the “villain” plot niche, but as the character developed, they kept turning out to be “too nice.” In other words, the author had a classic antagonist on his/her hands – someone who wasn’t really a bad guy, but another good person whose agenda and beliefs clashed with the protagonist’s desires. This generally lead to the author’s plot collapsing, because a non-villainous antagonist simply wouldn’t do some of the horrible things that the author’s initial plot required.
In both cases, the solution is twofold: first, the author has to recognize that even in the action-adventure genre, it is possible to write a gripping story that has no villain at all; second, the author has to think long and hard about the story they want to write, and look at how to do it without layering on a lot of assumptions about what “has to” be there.
SF has entire subgenres of stories that don’t require villains (though most of them can accommodate one, should the author decide that one of the characters is more evil than they had at first thought). First contact stories don’t require a bad guy; figuring out how to communicate with an alien species and how to understand an alien culture can provide plenty of problems without adding a bad guy to mess things up further. Stories of survival after a crash landing also don’t require a bad guy. Exploring new star systems and/or settling new planets can easily present sufficient problems and difficulties for characters without needing an actual villain, though in any sizeable group of humans there’s likely to be interpersonal conflict just because people disagree. Stories of discovery (whether of new star systems or of new knowledge) and invention cover the whole range, from the villain out to sabotage the exploration/research to the antagonist/rival who’s trying to beat the protagonist to the discovery to having no human obstacle or villain at all, because solving the problem is a big enough challenge all by itself.
There are also, in every genre, stories that focus on the growth of the character(s) – where the murder mystery or the quest for the unicorn scepter or the month-long cattle drive is the background and/or subplot to the main story about the character’s beliefs or relationships or personal growth, none of which involve a villain (other than the protagonist him/herself). These can be trickier to write than they sound, because even authors who understand the whole “no rules for writing, except do it and make it work” thing can get caught by their unconscious assumptions about what certain choices mean for a story.
For instance, take the author who has decided to write a story about a group of space marines. Practically the first thing the author looks for is what sort of war they’ll be involved in, or what mission they’ll be on. The story is about marines, so it’s obviously going to be action-adventure and involve fighting and a big battle as the climax, right? It’s so obvious and clear that nobody thinks twice about it.
But…why does this author want to write about space marines? What is it that’s about these particular characters? Maybe the author wants to examine the interpersonal relationships among these men and women, and the effect of long separation on their relationships with their families. Maybe the author wants to write a story about soldiers adjusting to a new peace, or performing a non-military mission like evacuating a planetary system whose star is about to go nova.
If the writer starts by asking questions like “who are they fighting/who are the bad guys? Where’s the big climactic battle going to be?” they’ll probably put a lot of effort into coming up with an action-adventure plot that may hang together, but that doesn’t really satisfy them, because the thing they considered the important thing about the story is the relationships, or what happens after the war, or the logistics of moving the population of an entire star system.
And this writer will almost never be happy with their villain, because having a villain at all is an artifact of the particular type of action-adventure plot the author has layered on top of the real story they want to write. If that sort of adventure is actually necessary to the story at all, in this case it needs to be a subplot…and we’re so used to action being central that it can be tough to relegate it to second place, even if one is consciously trying to do so.
Writers whose plot keeps collapsing because their “villain” is “too nice” are starting at the other end and working backwards to pretty much the same place. That is, instead of starting with a story and then developing an unnecessary action-adventure plot, complete with villain, they start with an action-adventure plot and develop characters that they can’t make fit. The two obvious solutions here are 1) change the characters or 2) change the plot.
Unfortunately, my experience has been that #1 seldom works. If the writer could have come up with different characters for this plot, she would have. If the character whom she assigned to the “villain” role has become too complex and/or interesting to behave as a stereotypical power-mad or money-hungry villain, I tend to see that as a Good Thing, even if it means making major changes to the plot.
In order to make those changes work, though, the author has to go through an extremely similar sort of thought process as the one who started with the characters and got off-track in the plotting: They have to think about what really interested them enough to tell this story in the first place, and whether that’s changed as they’ve developed the characters. They also have to think about whether having a more complex antagonist means radical changes to the plot (the original plan was for her to order an invasion for no particular reason, except that she’s eeeeevil, but now that she’s complicated, that just doesn’t work and the invasion will have to be jettisoned, along with everything that was supposed to result from it), or whether the grand outline of the plot can stay the same, but all the motivations and a lot of the details need to be reworked and deepened to fit the antagonist’s new personality (though finding a good, believable reason for the ex-villainess to order the invasion may be just as much work as redesigning the entire last half of the book to eliminate it).