So I have started work on polyphony and it is comes truly amazing. I was struggling to get user interface correct and then it just struck me that the easiest way for managing polyphony is to have each next note in the chord occupy next V/OCT input. Simple as it is. All you define is number of polyphony notes played at the time.
Amazingly great turned out that running 64 oscillators (4 notes chord times 16 oscillators per note in unison mode) in Wavetable mode just takes 40% of CPU time. The code is good and stable and allows to create better sound.