A word on video overlays

So I was looking around for alternatives to forcing my window to the very top. I discovered this project: http://www.codeproject.com/KB/directx/Overlay_Tools.aspx which was written in 2007 for C# using DirectDraw. (Kudos to ]Metty[ )

Sweet, the overlay stays on top, and that should work just fine, but wait, what's this?

It doesn't quite compile out of the box.

1) The project needs to be compiled for x86, not any processor or x86-64.

2) It uses unsafe code which is icky.

3) Doesn't play nice with Aero and Windows 7.

#1 is an easy fix. You can change that by adjusting the project settings.

#2 is as simple as replacing the unsafe code with a call to Marshal.Copy in the methods BlitYUY2 and BlitRGB32.

#3 is simple as well. Make a call to DwmEnableComposition in dwmapi to disable Aero.

Here are my changes in full to Overlay.CS

using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using Microsoft.DirectX;
using Microsoft.DirectX.DirectDraw;
using PixelFormat = Microsoft.DirectX.DirectDraw.PixelFormat;

namespace OverlayLib
{

    static class NativeMethods
    {
        [DllImport("kernel32.dll")]
        public static extern IntPtr LoadLibrary(string dllToLoad);

        [DllImport("kernel32.dll")]
        public static extern IntPtr GetProcAddress(IntPtr hModule, string procedureName);


        [DllImport("kernel32.dll")]
        public static extern bool FreeLibrary(IntPtr hModule);

        
    }

    /// <summary>
    /// A DirectDraw Overlay
    /// (c) 2007 Metty
    /// </summary>
    public class Overlay : IDisposable
    {
        #region ePixelFormat enum

        /// <summary>
        /// The PixelFormat of the render surface
        /// </summary>
        public enum ePixelFormat
        {
            RGB32, //Defines a RGB32 (R,G,B, -) pixel format
            YUY2, //Defines a YUY2 (16 bit) pixel format
        }

        #endregion

        private delegate uint Win32DwmEnableComposition(uint uCompositionAction);

        Win32DwmEnableComposition _myDwmEnable;

        private bool m_AlphaEnabled = false;
        private Surface m_BackBuffer;
        private Surface m_Buffer; //Render Source
        private Device m_Device;
        private OverlayEffects m_Effects;
        private OverlayFlags m_Flags;
        private ePixelFormat m_PixelFormat = ePixelFormat.RGB32;
        private Point m_Position = Point.Empty;
        private Surface m_Primary; //Display
        private RenderDelegate m_Renderer;
        private Bitmap m_RenderTarget; //Rendertarget

        private Size m_Size = new Size(100, 100);
        private IntPtr wdmFuncPtr;

        /// <summary>
        /// The position of the overlay on the screen
        /// </summary>
        public Point Position
        {
            get { return m_Position; }
            set
            {
                if (m_Device != null)
                {
                    Hide();
                    m_Position = value;
                    Update();
                }
                else
                {
                    m_Position = value;
                }
            }
        }

        /// <summary>
        /// The Size of the overlay
        /// </summary>
        public Size Size
        {
            get { return m_Size; }
            set
            {
                m_Size = value;

                if (m_Device != null)
                {
                    Dispose();
                    Initialise();
                }
            }
        }

        /// <summary>
        /// Boundings of the overlay
        /// Combins Position and Size
        /// </summary>
        public Rectangle Boundings
        {
            get { return new Rectangle(Position, Size); }
        }

        /// <summary>
        /// Called when the overlay shall be redrawn
        /// </summary>
        public RenderDelegate Renderer
        {
            get { return m_Renderer; }
            set { m_Renderer = value; }
        }

        /// <summary>
        /// The pixelformat currently used
        /// Either let this be determined default, or set it to try using the specified value as a start value
        /// </summary>
        public ePixelFormat PixelFormat
        {
            get { return m_PixelFormat; }
            set { m_PixelFormat = value; }
        }

        /// <summary>
        /// Determines whether alpha is enabled or not
        /// Beware: Is not supported on many (all?) nvidia graphic cards
        /// </summary>
        public bool AlphaEnabled
        {
            get { return m_AlphaEnabled; }
            set
            {
                m_AlphaEnabled = value;

                if (m_Device != null)
                {
                    CreateFlags();
                }
            }
        }

        #region IDisposable Members

        /// <summary>
        /// Disposes the overlay and frees resources
        /// </summary>
        public void Dispose()
        {
            if (m_Primary != null)
            {
                m_Primary.Dispose();
                m_Primary = null;
            }

            if (m_Buffer != null)
            {
                m_Buffer.Dispose();
                m_Buffer = null;
            }

            if (m_BackBuffer != null)
            {
                m_BackBuffer.Dispose();
                m_BackBuffer = null;
            }

            if (m_RenderTarget != null)
            {
                m_RenderTarget.Dispose();
                m_RenderTarget = null;
            }

            if (m_Device != null)
            {
                m_Device.Dispose();
                m_Device = null;
            }

            if (System.Environment.OSVersion.Version.Major >= 6)
            {
              
                _myDwmEnable(1);

                NativeMethods.FreeLibrary(wdmFuncPtr);
            }
        }

        #endregion

        /// <summary>
        /// Gets the directX pixelformat for the specified pixelformat
        /// </summary>
        /// <param name="e">ePixelFormat</param>
        /// <returns>DirectX PixelFormat</returns>
        public static PixelFormat GetPixelFormat(ePixelFormat e)
        {
            PixelFormat pixelFormat = new PixelFormat();

            switch (e)
            {
                case ePixelFormat.RGB32:
                    pixelFormat.Rgb = true;
                    pixelFormat.RgbBitCount = 32;
                    pixelFormat.RBitMask = 0xFF0000;
                    pixelFormat.GBitMask = 0x00FF00;
                    pixelFormat.BBitMask = 0x0000FF;
                    break;
                case ePixelFormat.YUY2:
                    pixelFormat.FourCC = 0x32595559;
                    pixelFormat.FourCcIsValid = true;
                    break;
            }

            return pixelFormat;
        }

        /// <summary>
        /// Initialises the overlay with the specified Boundings
        /// </summary>
        public void Initialise()
        {
            m_Device = new Device();
            m_Device.SetCooperativeLevel(null, CooperativeLevelFlags.Normal); //Only a overlay..

            //Create Primary
            {
                SurfaceDescription desc = new SurfaceDescription();
                desc.SurfaceCaps.PrimarySurface = true;
                m_Primary = new Surface(desc, m_Device);
            }

            //Create buffer
            {
                SurfaceDescription desc = new SurfaceDescription();
                desc.Width = Boundings.Width;
                desc.Height = Boundings.Height;
                desc.BackBufferCount = 1;
                desc.SurfaceCaps.Flip = true;
                desc.SurfaceCaps.Overlay = true;
                desc.SurfaceCaps.Complex = true;
                desc.SurfaceCaps.VideoMemory = true;

                //Try which pixelformat works
                do
                {
                    try
                    {
                        desc.PixelFormatStructure = GetPixelFormat(m_PixelFormat);
                        m_Buffer = new Surface(desc, m_Device);
                    }
                    catch (DirectXException)
                    {
                        m_PixelFormat++;
                    }
                } while (m_Buffer == null && Enum.IsDefined(typeof(ePixelFormat), m_PixelFormat)); //Stop if a valid format is found or no one is left

                if (m_Buffer == null)
                {
                    //Bad!
                    throw new GraphicsException("Could not create overlay - Either your graphic card does not support any of the available pixelformats or overlays in general.");
                }
            }

            //Create backbuffer
            {
                SurfaceCaps caps = new SurfaceCaps();
                caps.BackBuffer = true;
                m_BackBuffer = m_Buffer.GetAttachedSurface(caps);
            }

            //Create rendertarget
            {
                //We use a Bitmap, as it works on every graphicscard, a RGB-Surface wont
                m_RenderTarget = new Bitmap(Boundings.Width, Boundings.Height, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
            }

            if (System.Environment.OSVersion.Version.Major >= 6)
            {
                wdmFuncPtr = NativeMethods.LoadLibrary(@"dwmapi.DLL");



                IntPtr pAddressOfFunctionToCall = NativeMethods.GetProcAddress(wdmFuncPtr, "DwmEnableComposition");

                _myDwmEnable =(Win32DwmEnableComposition)Marshal.GetDelegateForFunctionPointer(pAddressOfFunctionToCall, typeof(Win32DwmEnableComposition));

                _myDwmEnable(0);
            }

            CreateFlags();
        }

        /// <summary>
        /// Creates the render flags
        /// </summary>
        public void CreateFlags()
        {
            //Create Flags
            {
                m_Flags = OverlayFlags.Show;

                if (AlphaEnabled)
                {
                    m_Flags |= OverlayFlags.Effects | OverlayFlags.KeySourceOverride;
                }
            }

            //Create Effects
            {
                m_Effects = new OverlayEffects();

                //Transparency
                if (AlphaEnabled)
                {
                    ColorKey key = new ColorKey();
                    key.ColorSpaceHighValue = key.ColorSpaceLowValue = 0; //Make Black (0,0,0) transparent
                    m_Effects.DestinationColorKey = key;
                }
            }
        }

        /// <summary>
        /// Renders (Updates) the overlay
        /// * Calls the RenderDelegate
        /// </summary>
        public void Update()
        {
            if (m_Device == null)
            {
                return;
            }

            try
            {
                if (m_Renderer != null)
                {
                    //Draw on the backbuffer
                    Graphics g = Graphics.FromImage(m_RenderTarget);
                    g.Clear(Color.Black);
                    m_Renderer(g);
                    Blit(m_RenderTarget, m_BackBuffer);
                }

                //Flip maps
                m_Buffer.Flip(null, FlipFlags.Wait);

                //Render



                m_Buffer.UpdateOverlay(m_Primary, Boundings, m_Flags, m_Effects);
            }
            catch (SurfaceLostException)
            {
                //May occur, but not bad - TestCooperativeLevel wont work in some D3D apps, but overlay will still render
                if (m_Device.TestCooperativeLevel())
                {
                    m_Device.RestoreAllSurfaces();
                }
            }
        }

        /// <summary>
        /// Hides the overlay from screen
        /// Can be undone by calling Update()
        /// </summary>
        public void Hide()
        {
            if (m_Device == null)
            {
                return;
            }

            m_Buffer.UpdateOverlay(m_Primary, Boundings, OverlayFlags.Hide);
        }

        /// <summary>
        /// Blits the RGB Bitmap to the specified surface
        /// </summary>
        /// <param name="src">RGB Bitmap</param>
        /// <param name="dest">Surface</param>
        public void Blit(Bitmap src, Surface dest)
        {
            switch (m_PixelFormat)
            {
                case ePixelFormat.RGB32:
                    BlitRGB32(src, dest);
                    break;
                case ePixelFormat.YUY2:
                    BlitYUY2(src, dest);
                    break;
            }
        }

        /// <summary>
        /// Blits the RGB Bitmap to a YUY2 surface
        /// </summary>
        /// <param name="src">RGB Bitmap</param>
        /// <param name="dest">YUY2 Surface</param>
        public void BlitYUY2(Bitmap src, Surface dest)
        {
            BitmapData ds =
                src.LockBits(new Rectangle(0, 0, src.Width, src.Height), ImageLockMode.ReadOnly,
                             System.Drawing.Imaging.PixelFormat.Format24bppRgb);
            try
            {
                LockedData dd = dest.Lock(LockFlags.WriteOnly);

                try
                {
                    int ps = ds.Stride - (ds.Width * 3);
                    byte[] pd = new byte[dd.Pitch - (dd.Width * 2)];



                    byte[] ptr = new byte[Math.Abs(ds.Stride) * ds.Height];
                    int ptrIndex = 0;
                    Marshal.Copy(ds.Scan0, ptr, 0, Math.Abs(ds.Stride) * ds.Height);

                    for (int h = 0; h < ds.Height; h++)
                    {
                        for (int w = 0; w < ds.Width; w += 2)
                        {
                            byte[] dbuf = new byte[4]; //2 pixel (2x16bit)

                            byte r1 = ptr[ptrIndex + 0];
                            byte g1 = ptr[ptrIndex + 1];
                            byte b1 = ptr[ptrIndex + 2];

                            ptrIndex += 3;

                            byte r2 = ptr[ptrIndex + 0];
                            byte g2 = ptr[ptrIndex + 1];
                            byte b2 = ptr[ptrIndex + 2];

                            ptrIndex += 3;

                            //Dont ask me for the conversion formulas - They are from FourCC and a bit of own modifications to match colors better
                            dbuf[0] = (byte)Math.Min(255, (0.230 * r1) + (0.600 * g1) + (0.170 * b1)); //Yo - luminescent 1
                            dbuf[2] = (byte)Math.Min(255, (0.230 * r2) + (0.600 * g2) + (0.170 * b2)); //Y1 - luminescent 2
                            dbuf[1] = (byte)Math.Min(255, +(0.439 * r1) - (0.368 * g1) - (0.071 * b1) + 128); //Ux - same for both
                            dbuf[3] = (byte)Math.Min(255, -(0.148 * r1) - (0.291 * g1) + (0.439 * b1) + 128); //Vx - same for both

                            dd.Data.Write(dbuf, 0, dbuf.Length);
                        }
                        ptrIndex += ps;

                        if (pd.Length > 0)
                        {
                            dd.Data.Write(pd, 0, pd.Length);
                        }
                    }
                }
                catch (Exception e)
                {
                    MessageBox.Show(e.ToString());
                }
                finally
                {
                    dest.Unlock();
                }
            }
            catch (Exception e)
            {
                MessageBox.Show(e.ToString());
            }
            finally
            {
                src.UnlockBits(ds);
            }
        }

        /// <summary>
        /// Blits the RGB-Bitmap to a RGB32 Surface
        /// </summary>
        /// <param name="src">RGB Bitmap</param>
        /// <param name="dest">RGB32 Surface</param>
        public void BlitRGB32(Bitmap src, Surface dest)
        {
            BitmapData ds =
                src.LockBits(new Rectangle(0, 0, src.Width, src.Height), ImageLockMode.ReadOnly,
                             System.Drawing.Imaging.PixelFormat.Format24bppRgb);
            try
            {
                LockedData dd = dest.Lock(LockFlags.WriteOnly);

                try
                {
                    int ps = ds.Stride - (ds.Width * 3);
                    byte[] pd = new byte[dd.Pitch - (dd.Width * 4)];

                    byte[] ptr = new byte[Math.Abs(ds.Stride) * ds.Height];
                    int ptrIndex = 0;
                    Marshal.Copy(ds.Scan0, ptr, 0, Math.Abs(ds.Stride) * ds.Height);

                    for (int h = 0; h < ds.Height; h++)
                    {
                        for (int w = 0; w < ds.Width; w += 1)
                        {
                            byte[] dbuf = new byte[4]; //2 pixel (2x16)

                            byte r1 = ptr[ptrIndex + 0];
                            byte g1 = ptr[ptrIndex + 1];
                            byte b1 = ptr[ptrIndex + 2];

                            ptrIndex += 3;



                            dbuf[0] = r1;
                            dbuf[1] = g1;
                            dbuf[2] = b1;

                            dd.Data.Write(dbuf, 0, dbuf.Length);
                        }

                        ptrIndex += ps;

                        if (pd.Length > 0)
                        {
                            dd.Data.Write(pd, 0, pd.Length);
                        }
                    }
                }
                finally
                {
                    dest.Unlock();
                }
            }
            finally
            {
                src.UnlockBits(ds);
            }
        }
    }

    /// <summary>
    /// Delegate for rendering
    /// </summary>
    /// <param name="g">Graphics Object</param>
    public delegate void RenderDelegate(Graphics g);
}

I hope this was useful.

Other changes may be to upgrade to directx 9 overlays (directdraw seems to work OK though..), or dwmapi http://msdn.microsoft.com/en-us/library/aa969540(v=VS.85).aspx

Thanks!

Another update to ChinMusic

Link!

This version fixes a problem where the graphics card doesn't handle Non-power of 2 textures.

There really shouldn't be that many, but the gfx card in my new machine doesn't. (What is this, 2005?)

Proposal for integrating IPC with emulators

When working on Chin Music, I've come to a number of conclusions.

* What I'm doing is not portable.

* What I'm doing clearly does not always work.

* What I'm doing relies heavily on exploiting some glitches in the system.

* What I'm doing only works with a handful of programs.

I've done some thinking and have come up with what I hope is a brilliant and elegant solution to the problem at hand.

The original idea was to be able to manipulate an emulator with no participation on the part of the emulator's development team. I could just put an emulator in a black box, send it window messages, and it would send a bitmap back. This means doing silly things like killing the menu and status bars, stripping the window decoration, and messing with the device context. All of which can have undesirable results.

An alternative idea is cooperation, either through direct integration or via forking the project to support duplex communication with external applications.

So here is the scenario playing out in my head.

For each emulator there would be two message queue. Queue 1 is the input queue. This is where messages such as button up, button down, cursor moves, volume is increased, game is muted would be sent from the front-end to the emulator. The emulator would consume messages from the queue.

Queue 2 is the output queue. This is essentially a broadcast of what's being displayed on the screen at a given time. The structure could simply be width, height, depth, endianess, followed by the the image generated by the emulator. The queue could be read, not consumed, by the front end. On a local system, this should at most generate one or two frames of lag. Copying a bitmap is far less intense than the operations being done inside the emulator.

One or the other queues could be used, both do not have to be.

Other than communicating with front-ends, this could be most useful for scripting AIs, screen casting, or any number of other applications.

This would also mean only having to configure a game pad or arcade stick from one application instead of editing multiple configurations. This could also mean making up for deficiencies in emulators which lack support for (as an example) taking a screen shot by holding the start and select buttons.


Lets examine what a sample syntax might look like for an NES system

<<system command> <value>>  |<<game command> <controller port> <value>>

system commands
* pause
* resume
* volume_up
* volume_down
* set_volume         X
* speed_up
* speed_down
* set_speed          X
* save_state         X
* load_state         X
* exit


player commands
* button_press    X A
* button_release X A
* button_press    X B
* button_release X B
* button_press    X Select
* button_release X Select
* button_press    X Start
* button_release X Start
* button_press    X Up
* button_release X Up
* button_press    X Down
* button_release X Down
* button_press    X Left
* button_release  X Left
* button_press    X Right
* button_release X Right

For more advanced emulators, you could set the delta of one of the sticks, or the movement of the mouse, etc.

Using Zero MQ (http://www.zeromq.org/) which satisfies cross platform requirements of many emulators, and boasts great speed seems like an ideal way to move forward.


It's my goal to soon release some prototype code to demonstrate the effectiveness and practicality of this solution. I would greatly appreciate developer and user feedback on this concept. Thanks.

Building a head 2 head box.

I wrote Chin Music EX to make any game playable head to head on a cabinet. I've decided to go the next step and make my own head to head arcade box.

I already have a 20 inch computer screen I haven't been using with integrated speakers. So that's a big part of the expenses.

Here is my projected parts list.

Motherboard CPU Combo
http://www.amazon.com/gp/product/B0041RSC94
 $75.25

The Dual Core Atom and the 3150 should be more than adequate for emulating most consoles and has good drivers for Linux should I choose to go down that route later.

RAM (4 gb)
http://www.amazon.com/Crucial-CT25664BC1067-204-PIN-PC3-8500-SODIMM/dp/B001KB6Z2U
2 x $12.99 = ~ $26

4 gb of ram should be way more than enough for emulating anything I want.

Controllers x2
http://www.amazon.com/Retro-Sega-Saturn-Classic-Controller-Pc/dp/B003KMWMYW/
2x 6.56 = ~ $14

I will be tearing these apart to get at the control boards. This tutorial here should get me on the right track.
http://www.slagcoin.com/joystick/introduction.html

I am still giving strong consideration to making the control panels modular, so you can pop off the controllers and play side by side in addition to head to head.

Buttons and Sticks x14, x2
http://www.ebay.com/itm/ws/eBayISAPI.dll?ViewItem&item=380222939700#ht_3199wt_1270
$40.00

I might get a few extra buttons in the future, but this should be more then plenty for the moment.

Pico 12 volt PSU (90 watt) + 12 volt PSU
http://www.amazon.com/PicoPSU-90-90W-12V-DC-DC-mini-ITX/dp/B00316RGD4/ref=sr_1_20?s=electronics&ie=UTF8&qid=1315934137&sr=1-20

http://www.amazon.com/Kinamax-AD-LCD12-Monitors-Adapter-Supply/dp/B000VE7GQQ/ref=pd_bxgy_pc_text_b

$31.00 + 9.19 = ~$40.00

90 watts should be more than enough for my needs. This machine won't have a DVD drive, or anything like that. I don't want any fans either if I don't need them. Also only has the bare minimum of wires.

Non-construction parts cost:  ~195.25

After I take some measurements, I will post about the actual construction costs.

Chin music in action

Chin music in action

Chin Music Ex : Strike 2

Download Link (Updated!)

Screen shot

A screen shot from an early prototype

Chin Music Ex 1.0

Chin Music 1.0

Link

ChinmusicEx (1.0)



A tool for manipulating window output.


Developed and maintained by Joel Longanecker (Joel.longanecker@gmail.com)


Requirements: 


* windows xp / 7
* .NET 4.0




Usage:


* ChinMusicEx.exe <path-to-program> <command-line-options>


ChinMusicEx.exe must reside in a directory with no spaces in the path. I'm working on this. So for now, place it somewhere like c:\ChinMusic\*


It's also helpfull to have the path to chinmusic.exe added to the PATH environment varible to simplify use. To quit chinmusic, use q. To rotate the window, use r. These are hardcoded for now. All other settings are editable in ChinMusicEx.exe.XML


Make sure the emulator you are working with meets the following criteria. Tweak as needed.


It is preferable that the emulator writes to a GDI bitmap rather than directly to the screen. You may need to use older plugins. The project 64 directx 6 plugin works fine, the directx 9 one does not.


The emulator must run in windowed mode, full screen can really break shit bad.


Your task bar might dissapear and not come back. If that happens, just run ChinMusicEx again, and quit with q.


Joystick input works... As far as I have tested.


Your feedback is important. Tell me what works and what doesn't.

Obligatory first post

So, my name is Joel and I like writing software. I acknowledge that there is a much more famous Joel who writes about software. I am not him.

I like working on games and small quick weekend projects. Sometimes I'll do some home-brew, or talk about working in the corporate world.