First the good news. I finished the first draft of my novel, Overkill!
The bad news is that it needs work. A lot of work. That’s not atypical for a first draft, but it’s still daunting.
First there’s the global issues (this might sound bad but bear with me). I struggled to get to the end of this draft. I started to lose enthusiasm for the story and I probably would have given up if not for reviewing the opening chapters, which still draw me in and get me excited. That battle is evident in the manuscript. As it stands ending is disappointing. At some point I just decided to get to the end and fix it later, and it shows.
In my critique group, I’m reading just a few scenes from the end. My fellow writers expressed some surprise that this is the case and made some really great suggestions on how I can get things back on track. I’m confident that applying these suggestions will not only carry the last act, but will improve the work throughout.
You’re probably wondering what this has to do with programming. I’m planning to do a pretty rigorous analysis of the manuscript as I go to the second draft. This is going to involve a lot of the process described in Shawn Coyne’s The Story Grid, John Truby’s The Anatomy of Genres, numerous resources on story structure, and a number of other attributes of my own making. The process involves gathering metadata on each scene and then examining it, to make sure the scene works as a unit, that the scenes have rhythm and flow together, and that the whole story works in the Science Fiction genre. One of the ways this is usually done is to create a spreadsheet, with each scene as a row and all the structural elements as columns.
I already know a spreadsheet won’t work for me, because I won’t be able to maintain it as I make changes to the story. I’ll create the sheet, iterate through a set of changes and then balk at updating the sheet and abandon the process. Over the years I have adopted countless organizational systems like this, many of them excellent. I have abandoned them all because I find the maintenance so difficult. The only way this is going to work for me is if the metadata is maintained in the scene itself.
Up until now I’ve been using Scrivener for writing. It’s a great little application, but I have two issues with it. For all its features, there’s no way I know of to attach the sort of metadata I need to each scene. I tried creating a set of parallel documents, but it quickly became clear that this will hit the same snag as maintaining a spreadsheet. It will start well then fall apart. I’ll start getting down on myself for not doing that and that will sap some fun out of the process, making it ever more likely that I walk away frustrated. Nobody wants that.
My other problem with Scrivener is tangential: there’s no longer a Linux version. I expect to make the transition to all-Linux all-the-time in the not too distant future, and although there are reports that it will run under WINE (a Windows emulator) my attempts to make it work have failed. Either way, it’s a “brittle” solution: an update in Scrivener or any of the tools that make it run could cause it to fail. That means spending time trying to resolve esoteric software issues instead of writing. No thanks.
While looking for a Scrivener replacement, I found NovelWriter, an open source tool that does almost everything I use in Scrivener. It’s not a plug-in replacement, but the differences are small enough that it’s a viable alternative. Better yet, it’s written in Python, a language I’ve got some experience with. Documents in NovelWriter are written in plain text, using Markdown for formatting. Markdown is dead simple to master. Better yet, NovelWriter already has some scene level metadata!
I thought I’d try a proof-of-concept, so I downloaded the source code, added support for my scene data and a facility to export it to a spreadsheet, and it worked! I shared my idea with NovelWriter’s author, Veronica Berglyd Olsen, in a GitHub discussion thread and together we came up with a plan to implement my idea, her ideas, and a couple of other ideas/feature requests. Olsen has been actively working on the project and I’d expect these features to show up soon (in open source, soon can mean a week to a couple of years). But the timing doesn’t matter because now I know how to add metadata that will be compatible with the new version and can build the capability into a fork if I need to. That’s the power of knowing how to code this stuff. It’s also the exact peril! I’ve just spent a couple of days working to implement the new feature, except I don’t know the code well enough to do it in a way that does everything the specification asks for and I can see getting bogged down in coding. But I’m not supposed to be coding, I’m supposed to be working on the manuscript!
Because a NovelWriter project is all in standardized files instead of some closed proprietary format, in the worst case I can write stand alone code that extracts the metadata and stuffs it into a spreadsheet. Even if Olsen decides the whole thing is a bad idea (which I don’t expect at all), I can still do what I want. This seals the deal, I’m porting to NovelWriter right now!
Moving takes a little work. Scrivener has an export option, but not to a useful format. The manuscript is stored in an opaque file format. In order to share with my critique group, I’ve been copying scenes, pasting them into a LibreOffice Writer document, and then into our shared folder. But neither Scrivener or LibreOffice support export to Markdown. Enter Pandoc, an incredibly powerful open source file conversion tool that will take an ODT file and transform it to several different dialects of Markdown. Perfect.
Make that not quite perfect. NovelWriter’s Markdown has a couple of quirks, as does Pandoc. I can get most of the way there but there’s some tedious manual clean-up required. At this point I fall back to a short PHP script that takes the Pandoc Markdown output, fixes the quirks and writes it out in something NovelWriter can import cleanly. The only manual work is to copy across some notes I’ve made in scrivener, typically one or two per scene, not a problem.
So there it is. Knowing code is both something that allowed me to have a significant input to the direction of what is now my main writing tool, and something that could easily suck up a big chunk of time spent not writing. It is oh so tempting to solve every problem with code, but there’s lots of times when doing it yourself is not the best idea.