I found this bit of information really helpful
"There is a theory of writing called the Circle of Story Elements. It states that there is a circle of 5 story elements that a writer must work through before you have created a complete story. You may start at any point on the circle, but you have to get to all of them before you're done. The circle goes like this:
characters - character dynamic - plot - atmosphere - wordbuilding - then back to characters again."
I tried that starting with worldbuilding and the story is coming along great so far. I'm surprised I've gotten only one reply. (Thanks for replying frazzle)