I noticed the Xvnc process and figured this was a desktop to which you needed remote access (for whatever the reason). For that you can have the X11 module for Xvnc installed to see the :0 server. If you just need to access X remotely on it form time to time in Xvnc, run a very lightwaight WM (like Minimal Window Manager - mwm) and avoid the bloating of fancy WMs. A really old and quite spectacular WM was AmiWM, for those of us who grew up on Amiga. Very stable, very versatile and lightweight with the Workbench look. Written by a Swede in the 90s. http://xwinman.org/amiwm.php FVWM2 has been rocking my world for decades now. You can explicitly set processor/memory affinity in most cases manually from /proc but there are CLI tools for this. It really depends on what performance hit we are talking about, although all CPU cycles is most certainly a lot. A lightweight WM would be my bet. PS> I must say I love Xlib talk. I am writing a lightweight VNC replacement that I want to use for a special purpose. It will be opensourced. All I do is segmented X framebuffer copies. Long live MIT's Athena project.