Hi Michael,
I am certain you will find an elegant solution for this.
I really like the UI so far.

One thing that is worth considering is having a small + or other symbol on items that can also be right clicked.
(or as in Modo, where the icons change if you press ctrl or shft, showing you what other commands are available).