Prune is a 2D game engine I am building in which the editor and engine exist within the same runtime.
One editor, not three
One of the first major problems I needed to solve with the early prototypes was ensuring all the scene types were first-class citizens in the editor. I have spent a considerable amount of time ensuring that Prune is not three game editors disguised as one. Each of the scene types builds upon the same WorldScene foundation, they share the core editor shell and then add their own behaviour, tools and object semantics.
Although the work was slow, this part of the design was relatively straightforward to solve, I had a vision of what I wanted and I was able to implement it.
What I wasn’t prepared for was how hard it would be to design the editor, both in thinking time and development effort.
Unexpected design issue
I have recently started working on the tooling. Prune has moved beyond the simple creation tools for each scene type. I need to start adding the interaction tools that users expect from a 2D game engine editor. I am starting with Select and Move, with Scale planned for sometime next week.
I mistakenly thought this would be quite simple, I already had tools; I just needed to add a few more buttons.
Over the last couple of weeks I have gone through three separate versions of the tool palette design before settling on the currrent design.
Finding the solution
I started with the tool palette below the main toolbar. It worked but didn’t feel right, the tools themselves were disconnected from the rest of the editor.
I then tried moving them into the same panel as the scene tools. That was wrong straight away, it made the scene-type panels too big. It introduced scrolling, which meant the scene specific controls were not visible, this is an existing problem which the additions made much worse.
The answer in the end was a floating tool panel above the viewport. I settled on the top right, the tools are near the other panels but separated from the inspector controls.
It looked great, I thought I had solved the problem but it raised a major usability issue. The tool palette was a separate panel, clicking the viewport brought the viewport forward, hiding the tools behind the viewport.
After some research the fix was to stop treating the tool palette being its own Dear ImGui panel. I needed to render it as part of the viewport itself. The tool palette is an overlay on the viewport, it is rendered above the scene and excluded from viewport hit-testing.
Dual-mode tool palette
The tool palette had two responsibilities. The palette houses the general-purpose editor tools and any creation tools specific to the scene type. For example, adding a platform in the platformer scene type or a wall in the simple shooter scene type.
The top half of the palette contains interaction tools. You enable a tool such as Select or Move, and it remains active until you choose another tool.
The bottom half is immediate actions, you click and Prune creates an entity, records it in undo history, and leaves the selected interaction tool unchanged.
I expect this design to work well moving forward. Most of the features in game engine editors are controls and flags in the inspectors, not buttons in the tool palettes
The current palette is tiny but it gives Prune a place for additional interaction tools, scale and rotate are next on the list.
The tool palette also has the benefit of tidying up the scene type panels, I was able to move buttons out of the panels reducing all their heights.
