Implicit Resource management Design–Idea

An idea came into my mind today. Why not just make an implicit(automatic) resource management system instead of having some kind of an external resource management system.

Ever since the way I did it was having an external resource manager that I would then use to create the resource I want. Take a look at this sample code below.

Code Snippet 1

1. void FunkyFunkFunk( void )
2. {
3.     Texture *pTexture = TextureManager::GetInstance()->CreateTexture( … );
4.     
5.     // Use texture
6.     pTexture->DoSomething();
7.     pTexture->DoMoreStuff();
8.     pTexture->Render();
9.  
10.     // We're done with the texture. Remove texture from the manager
11.     TextureManager::GetInstance()->DeleteTexture( pTexture );
12. }

As you can see, we create a texture through a texture resource manager called TextureManager and once we’re done with the texture we go through the TextureManager again to remove that created resource.  It’s not really much of a big deal but what if we could just hide the resource management code stuff yet still having all the benefits and convenience of managing your resources? That’s where I thought and said, “Why not just place the manager inside the resource class instead and handle the resource management creation and deletion through the constructor and destructor of the resource respectively?”. hmmm….

 

– MrCodeSushi

A Simple Data Creation Framework

First of all, I wasn’t really sure if I wanted to write this post or not. The data creation framework I made is mostly hacked without proper/formal scripting architectural design. But if this article could help anyone out there in some ways in their programming tasks, even just a little bit, then this blog wouldn’t be a waste. With this in mind, this article will be very brief with very little explanation for its implementation.

This data creation framework I will present enables a data driven architectural design to deliver game data/assets to your game. Since this is a data creation framework, its main purpose is only to bring data; nothing more, nothing less. That means that although the script looks like a full blown scripting language, it is not. This is not like LUA or any other scripting languages that you can define procedural-scripting or gameplay code. This framework simply defines a set of data with an assigned values for the framework to easily read in the data.

If you take a look at the class definition[1], you’ll see lots of containers supporting each data type that the framework supports. Let’s say for example you have a sample script code that looks something like in Code listing 1.

Code Listing 1

1.     
2.     Vector3 vec3 = ( 1.0, 3.0, 2.578 );                            // 3D Vectors
3.     

Code Listing 1: An example of declaring a Vector3 data with a defined value of (1.0, 3.0, 2.578)

vec3, as Vector3 as its data type, will be stored in the Vector3Array container.

So this framework is very simple and straightforward. It doesn’t do anything else other than declare data and this will do fine for the system I’m currently working towards on. The main methods that does the main string parsing and reading are in the GetLine, LoadFromFile, ReadSection, and ReadValue methods.

Here’s the implementation for the framework[1][2]. And a demo script on what the framework can do(Code Listing 2).

Code Listing 2

1. // This is a sample script used to test Pulse Engine's PulseScript system.
2. // Below shows some of the features already available.
3.  
4. // A comment. An empty space follows below.
5.  
6. /*                        MULTI-LINE COMMENT
7.     This is a multi-line comment. Multi-line comments
8.     can span multiple lines and PulseScript will strip
9.     it all out until it hits the closing mult-line comment character.
10. */
11.  
12. // Another comment
13.  
14. Section pass // Comment in the same line as the code.
15. {
16.  
17.     String string = "PulseScript can take in strings!!!!";
18.  
19.     String multiLineSting = "PulseScript can also take in multi-line strings!!!!\\par                                 This is the second line of a multi-line string.\\par                                 Third line of the multi-line string. Another string in the third line. \\par                                 Fourth line of the multi-string. This is the end of a multi-line string.  ";
20.     
21.     // Supports boolean values
22.     Bool bool1 = true;
23.     
24.     // Supports scalar types (i.e. int and float)
25.     Float f = 3.987654321;
26.     Int n1 = 5;
27.     Int n2 = 10.9257; // This will automatically convert the value to int(floor, so the value will be 10).
28.         
29.     // Supports section comments
30.     Bool /*Yup, this is a section comment*/ bool2 = false;
31.     /*Another section comment*/ Bool bool2 = false;
32.     Bool /* section comment */bool3 /*another section comment*/ = true/*yet another section comment*/; /*and another*/ // And another…
33.     
34.     // Supports vectors too!!!
35.     Vector2 vec2 = ( 1.0, 3.0f /*appending 'f' is optional*/ ); // 2D Vectors
36.     Vector3 vec3 = ( 1.0, 3.0, 2.578 );                            // 3D Vectors
37.     Vector4 vec4 = ( 1.0, 3.0 , 2.578, 3.14159265 );            // 4D Vectors or Quaternions
38.     /* If you haven't noticed I just pulled a pi there. lolz */
39.     
40.     /* Supports recursive sections (sections inside a section) */
41.     Section Mesh
42.     {
43.         Section Material
44.         {
45.             String mergeMode    = "Blend";
46.             String bumpMap        = "bump.tga";
47.             String textureName    = "MeshTexture.tga";
48.         };
49.         
50.         String meshName = "tiny.x"; // The infamouse DX character
51.     };
52.     
53.     // Array of ints.
54.     Int[5] indices1 = { 1, 2, 3, 4, 5 }; // We can explicitly define the size of an array
55.     Int[9] indices2 = { 1, 2, 3,    \\par                         4, 5, 6,    \\par                         7, 8, 9,    \\tab \\ comma at the end is optional
56.     };
57.     // NOTE: Another important thing is that only arrays and strings can define values in multiple lines.
58.     Float[3] floatArray = { 3.256, 1.23, 456.789 };    // Float arrays
59.     
60.     Vector2[2] vert2dArray1 = { (3.0, 2.0), (2.56, 1.23) };
61.     Vector2[3] vert2dArray2 = { (3, 2.0), (2.56, 1.23),    \\par                                  (1.23, 4.56) };
62.     /*
63.      You can't do things like this though:
64.         Vector3[2] bla = { ( 1.2, 2.2, \ // <— Incomplete
65.             2.1 ), (1,2,3 )};
66.     */
67.     // TODO: We still need to implement these…
68.     // Same variable name implicitly overrides the older one. Do something about this.
69.     // Int[] indices3 = { 6,7,8,9,10 }; // We can probably do something like this in the future…
70. };

Code Listing 2: Sample data creation script. Download it here[3]

 

Bibliography:

1. PulseScript.h, http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Include/PulseScript.h

2. PulseScript.cpp, http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Source/PulseScript.cpp

3. Sample PulseScript script code, http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Demo/SampleScript.ps

– MrCodeSushi

Data Creation Framework Screenshot

A preview on a Data Creation Framework/Scripting System I’ve been working on the side. Not bad considering I only work on this on Sundays IMO. :p An article discussion will follow soon!

Sample Pulse Script

Game System Registry With Integrated Simple Unit Testing Framework

This system I will share today is roughly adopted from Guillaume Blanc’s Registrer(Registry) system[1]. This Registry system is designed to keep track all registered classes and objects in an easy, global, and manageable manner. This system can easily send events such as initialization, shutdown, unit testing, or anything else that you can imagine and each registered systems/objects can react to a specified event. If this sounds very familiar to you, chances are you’ve already heard this in your Design Patterns class. Yes, this is the dispatcher-listener design pattern. An object or a system (aka listener) registers to the Registry(aka dispatcher) and when the Registry signals a message, it sends a message to each and every registered listeners.

There are two types the Registry accepts. One is a class (static classes) and the second is an instantiated object. An interface class will play a big role on these two types in order for them to be accepted by the Registry. Code listing 1 shows the interface class.

Code Listing 1

class IRegistryObject : private NonCopyable
{
public:
    // Initialize object priority
    IRegistryObject( ERegistryPrimaryPriority::Type primaryPriority, ERegistrySecondaryPriority::Type secondaryPriority, BOOL bRegisterObject = TRUE );
    virtual ~IRegistryObject ( void );

protected:
    friend Registry;

    // Registry object operation handling
    virtual BOOL DoRegistryOperation( ERegistryOperation::Type op ) = 0;
    // Less than registry object comparison
    static BOOL IsLessPriority( const IRegistryObject *plhs, const IRegistryObject *prhs );

private:

    ERegistryPrimaryPriority::Type m_primaryPriority;
    ERegistrySecondaryPriority::Type m_secondaryPriority;
};

As you can see in Code Listing 1, the abstract base class needs the derived classes to define DoRegistryOperation(). DoRegistryOperation() method is the entry point for all the events sent by the Registry.

You may also have noticed that the constructor accepts two parameters named ERegistryPrimaryPriority and ERegistrySecondaryPriority. These enums defines the priority of the Registry Object from lowest to highest priority. These priorities are very important as they control the order of the registered objects to call. What you can do is probably set very important systems or classes having the highest priority without dependencies. Less important systems or systems that requires dependencies would obviously need to be set in a lower priority than the dependency it requires.

For registering an instantiated object, one would simply inherit the class with IRegistryObject then calling IRegistryObject’s constructor to automatically register it in the Registry.

For classes, it takes a little bit more of explaining to do. The problem with this is that, there are no actual objects to register in the Registry. So what we would do is create a class that encapsulates or connect the class in question in some way. Thus, we devised a macro to do exactly like that. Take a look at Code Listing 2.

Code Listing 2

#define PSX_START_REGISTRY_OBJECT( _class_, primaryPriority, secondaryPriority )    \\par     class RegistryObject##_class_ : public IRegistryObject    \\par     {    \\par     public:    \\par         RegistryObject##_class_##( void ) : IRegistryObject( primaryPriority, secondaryPriority ) { } \\par         virtual BOOL DoRegistryOperation( ERegistryOperation::Type op );    \\par     };    \\par     static RegistryObject##_class_ gRegistryObject##_class_##;    \\par     \\par     BOOL RegistryObject##_class_##::DoRegistryOperation( ERegistryOperation::Type op )    \\par     {    \\par         switch( op ) \\par         {

#define PSX_END_REGISTRY_OBJECT( )    \\par         default:    \\par         break; \\par         }    \\par         return TRUE;    \\par     }

#define PSX_REGISTRY_ON_INITIALIZE()    break; case ERegistryOperation::INITIALIZE:
#define PSX_REGISTRY_ON_SHUTDOWN()        break; case ERegistryOperation::SHUTDOWN:
#define PSX_REGISTRY_ON_TEST()            break; case ERegistryOperation::TEST:

Quite scary, no? Well, don’t be. These are just predefined preprocessors that basically replaces everything in code that finds the same name as the defined name. What these macro does is that it declares a new class, derived from IRegistryObject , and defines the DoRegistryOperation() method. The DoRegistryOperation() is actually empty and you’ll be the one to write code into it let be initialization, shutdown, or test code using the PSX_REGISTRY_ON…() macro declarations. Code Listing 3 shows a sample on how the macros are used.

Code Listing 3

PSX_START_REGISTRY_OBJECT( String, ERegistryPrimaryPriority::Normal, ERegistrySecondaryPriority::Normal )
    PSX_REGISTRY_ON_TEST()
    {
        String str1 = "Hello World.";
        PSX_Assert( str1.GetLength() == 12, "Length should be 12." );

        SIZE_T index = str1.FindFirstOf( "l" );
        PSX_Assert( index == 2, "Index should be 2." );

        index = str1.FindFirstNotOf( "H" );
        PSX_Assert( index == 1, "index should be 1." );

        String str2 = str1.SubString( index );
        PSX_Assert( PSX_StrCmp( str2.GetCString(), PSX_String("ello World.") ) == 0, "Substring method failed." );

        return TRUE;
    }
PSX_END_REGISTRY_OBJECT();

Code Listing 3 shows registering a String class to the Registry and defining a Unit Test code for code testing.[2] Please study Code Listing 2 and Code Listing 3 to fully grasp and understand what it is trying to do.

And lastly, I present to you the Registry class shown in Code Listing 4.

Code Listing 4

class Registry
{
public:

    static BOOL ExecuteOperation( ERegistryOperation::Type op );

private:

    static void RegisterObject( IRegistryObject *pObj );

    static void UnregisterObject( IRegistryObject *pObj );

private:

    friend IRegistryObject;
    friend RegisterClassRAII;
    friend class TestRAII;

    // Custom compare trait for sorting registry objects based on their priority
    template < typename T >
    class RegistryObjectLessThan
    {
    public:
        BOOL operator () ( const T &lhs, const T &rhs ) const
        {
            return IRegistryObject::IsLessPriority( lhs, rhs );
        }
    };

    typedef SortedLList< IRegistryObject *, RegistryObjectLessThan<IRegistryObject *> > ObjectList;

    static ObjectList *m_pObjectList;

};

There’s nothing much really to discuss here. Registry objects simply register themselves by calling Registry::RegisterObject and they will be registered in the Registry. UnregisterObject removes them from the Registry.

Last Thoughts and Further Improvements

You may have noticed that the Registry class uses a SortedList (SortedLList). It is basically a sorted linked-list. That being said, it won’t be a good idea to register lots and lots of instantiated objects. This system is designed to handle application/game systems at best. Such systems would be a memory manager, Sound System, Input System, Grpahics System, etc… If you want to make it handle a great amount of instantiated objects then you may have to change a more capable container such as a vector, hash-table or a map for example.

Implementation

Full implementation of the Registry system/class can be found here[3].

Bibliography

1. Guillaume Blanc, Article source code, Registrer, Cross platform rendering thread, ShaderX7, 2009
2. Pulse::String class, http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Include/String.h, http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Source/String.cpp
3. Pulse::Registry class, http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Include/Registry.h, http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Source/Registry.cpp

Don’t forget a good dose of Mr. Louis Armstrong song!

What a Wonderful WOrld–Louis Armstrong

 

– MrCodeSushi

Your Own custom abstract IOStream interface

This is more of an obvious avoidable problem than a necessary work/solution. But I needed this on how I was working on my own little “engine”. A couple of months ago, I was working on a Virtual File System. I later discovered that std::fstream has a 4GB limit. So I resorted to using the Stream I/O instead which can handle greater than the 4GB limit. The problem with this is that it isn’t structured in an Object-Oriented way. Yes we want everything to be Object-Oriented. That being said, I ended up wrapping the Stream I/O into a neat simple to use class[1][2].

Now I’m working on a new system which requires a generic way to input and output streams. What I mean is that I want my system to not care where the system is reading in or writing out data. Let it be reading in from an input peripheral, reading from a file, writing to a console or writing to a file. So I designed a very simple abstract I/O stream class. It is broken down into two classes, IStream and OStream, which is derived from a base Stream class[3].

These abstract classes are definitely far from complete. But it seems to serve well for my purpose. So I can at least say that this is a good start.

If you take a look at the implementations provided, you’ll notice that only the FileIO class is currently derived from it right now. I still have to add a bunch of derived classes to encapsulate cin, cout, and a bunch of others that I can think of which is currently in my TODO list that I have to do sometime in the future(no pun intended).

Links:

1. FileIO.h – http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Include/FileIO.h

2. FileIO.cpp – http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Source/FileIO.cpp

3. Steam.h – Stream, OStream, IStream – http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Include/Stream.h

Dashing Song to Listen to:

Andrews Sisters–I’ve Got A Guy In Kalamazoo

A Flexible Material System in Design by Maxime Beaudoin

 

I was looking for some good Material System designs in teh interwebz and I stumbled upon this one article by Maxime Beaudoin. Quote from ShaderX6 book:

“Maxime Beaudoin Maxime Beaudoin is a 3D graphics programmer at Ubisoft Quebec studio, Canada. He started to learn real-time 3D graphics by himself in 2001 and received his B.S. degree in computer science in 2003. Since 2005, he has been working for Ubisoft during the day and developing his own next-gen game engine with some friends during his free time.”

 

Unfortunately, I don’t have a copy of ShaderX6 book nor I can download one from the internet. But I found a ppt presentation that he made for GDC 2008.

http://www.coretechniques.info/2008/Material.zip

P.S. Something I found really2x cool.

Back of the Mike (1938)

Back of the Mike is a short film done in 1937 for the Chevrolet Motor Company depicting the behind the scenes look at the making of a Western radio show. This documentary shows the various ways sound effects are created during the broadcast. Rain was created by pouring sand over a spinning potters wheel which sent it down a metal funnel onto a microphone which was covered by a paper bag. Fire was created by wadding up plastic wrap close to the mike.

16.1 MB’s. Video will play in the flash player below.

http://www.oldradioworld.com/media/Back_of_the_Mic_1938.swf

http://www.oldradioworld.com/Back_of_the_Mike.php

 

Cheers!
– CodeSushi

ResourcePool design.. AND ResourcePoolItem AND StoragePool AND Storage…

Man it’s been a while since the last time I’ve worked on my personal project. Been busy with work lately. Anyway, back to my resource manager.

I’ve been really productive in my project for the past 2 days and I think I have got a lot of things done. But I’m still far from finishing my resource manager. I’ve finished implementing all the base component of my resource manager and what is left is to actually work on THE Resource Manager itself. Although before I present the design, I would like to give credit to this one author who really helped me in designing my manager. I need to give credit where credit is due! His name is Greg Snook. I don’t have direct affiliation with him but I stumbled upon his book in my book collection recently, and his engine presented in his book contains A LOT of invaluable treasures! One of them is the design of the resource manager and while I was in the process of designing mine, I found out that my general design somewhat aligns with his. So I decided to use his as a general guideline instead.

Below I present an insanely tangled UML Class Diagram on how the sub-components are designed. The base storage class that holds the resource is StorageData. This acts like a static vector class. Another class is derived from it called StoragePool where it is composed of a list of StorageData. Thus StoragePool acts as a dynamic container that dynamically extends the list if StoragePool gets pool and vice-versa. Since StoragePool is a generic Storage Pool class, I’ve developed another class the simply extends and formalized the methods for resource handling and using the StoragePool(composite) class.

Ultimately, the StoragePool class will be used the ResourceManager which I haven’t finished working yet. For now I present the UML diagram and the links for the source code.

Enjoy!

StoragePool.h: http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Include/StoragePool.h
ResourcePool.h: http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Include/ResourcePool.h
ResourcePool.cpp: http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Source/ResourcePool.cpp
ResourcePoolItem.h: http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Include/ResourcePoolItem.h
ResourcePoolItem.cpp: http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/Source/ResourcePoolItem.cpp

Bit Flags Class

The more I flesh out my resource manager design the more I realize just how much more work and classes I need still need to add. While I was working on my ResourceItem class, one of the components it needs is to keep track of a bit field to keep track of its state whether it is loaded, ready to use, disabled, etc.  Instead of normally hacking my way through the code implementations i decided to encapsulate that bit field implementation instead. And I created a class that wraps bit manipulation called BitFlags. Here’s a simple UML Class Diagram below.

 

BitFlags member and interface.

Basically, this class handles the low level bitwise manipulation like setting which particular bits on and off. The interface seems self-explanatory so I’ll just let you figure the methods out. Link to the implementation is provided below. As an added bonus, here’s a bunch of bitwise manipulation macros extracted from my Pulse Library project.

1. // Bit manipulation macros
2. #define PSX_GetBit(x)                (1<<(x))
3. #define PSX_TestBit(flag, bit)        (((flag) & PSX_GetBit((bit))) !=0)
4. #define PSX_TestBitAll(flag, set)    (((flag) & (set)) == (set))
5. #define PSX_TestBitIfAny(flag, set) (((flag) & (set)) !=0)
6. #define PSX_SetBit(flag, bit)        ((flag) |= PSX_GetBit((bit)))
7. #define PSX_SetAll(flag, set)        ((flag) |= (set))
8. #define PSX_ClearBit(flag, bit)     ((flag) &= ~PSX_GetBit((bit)))
9. #define PSX_ClearAll(flag, set)     ((flag) &= ~(set))

 

Link to BitFlags class implementation:
http://codaset.com/codesushi/pulse-tec/source/master/blob/Source/Engine/BitFlags.h

Feel fre to ask if you have any question!

– CodeSushi

Resource Manager Design Implementation And A Little Update

It has been a while since my last post. After my little adventures on graphics programming, I’ve been quite busy with work lately so I haven’t been able to continue working on my little Pulse Library. Although, I’ve been thinking on what to move on next with my library for a while now. I have two choices. First, either work on the 2d stuff for now. Basically setting up basic rendering and hand written sprite system. This will also be the foundation for one of the things I’m also planning in the future: GUI or Graphical User Interface system. The second choice is simply to go directly to 3D first. Since DirectX pretty much works on 3D it would be easier for me to start on this direction. BUT WAIT! While I was too busy thinking about which way to go first, I totally forgot one really important thing! A resource manager! I can’t just start  writing all these systems up and use resources like crazy. I need a way to properly manage and dispose resources. I have already started writing about half of the manager last weekend and I’m hoping to finish it by the end of this week so I can finally ACTUALLY start working on my render ( then figure out on which direction to go… 2D or 3D ).

I’ll briefly discuss the manager in here for now and I’ll be posting another blog post discussing how my manager was designed and how it is used in detail. It would be nice if I can display pretty nice UML pictures in here but I’m lazy. So deal with it! I should start using UML next time though… Actually I just forgot how to use it and I need to review my software architecture class regarding on how to use it again. 😛

So here’s how my Resource Manager goes:

The whole manager uses a monolithic management structure that consists of a number of resource management subsystems(we’ll get to this later). This system is designed to have a one and only instance used throughout the time an application is run. That means we’ll be using a singleton design pattern.

When i say monolithic management structure, this system will literally handle ALL resources used by the application. Let it be partial streamed packets for sound and or music, game data objects (game data will be stored in a resource cache), graphics resources like textures, vertex and index buffers, shader effects etc. Yes, ALL TYPES OF RESOURCES.

Okay, so the problem with this approach is that how the hell will we able to do that?! Different types of resources requires radically different types of interfaces and if we merge them all to one, it’ll just result in one huge convoluted type of manager class interface that more than half of its methods doesn’t make any sense. In order to fix this problem, we need to study each resource type and try to derive a number of methods that’s pretty much being used by all of them. So what we need to do is basically build a VERY GENERIC type of resource base type. In my current implementation, I simply call it as a ResourceItem. The ResourceItem class contains abstract methods that derived classes will need to define. Some of its methods will be like Create(), Destroy(), Disable(), Restore(), LoadResource(), SaveResource(), etc…

So while the manager hands out different types of resources  based on our needs, internally, the manager doesn’t care much about what type of resource it is. All it cares is that its handling a ResourceItem and all the basic methods that it needs to create, maintain, and finally destroy are there. Simply as that!

This pretty much lays out the general idea of the manager that we will be using. Now for the resource management subsystem that I mentioned at the start, this is simply a storage pool. A storage pool is a container I specifically designed to store game resources. It behaves like an array that issues handles to new added resources. This storage pool is also a dynamic container that increases its storage size whenever needed. The difference is that, due to the huge performance hit by arrays, it uses a linked-list instead. Now we all know that linked-lists suffers huge cache misses during traversal. So the storage pool that I will be using uses a different trick. Instead of using linked-list per object, we use linked-list to link chunks instead. The keyword here is chunk. We allocate chunks or blocks containing a number of objects and when that chunk is full, we then allocate a new chunk and link the new one at the back of the list. Isn’t it cool? Such a simple design now actually has the best of both worlds(array and linked-list)!

Diagram showing a storage pool creating a new block or chunk then adding a new resource element.

 

Well, that’s all for now. Sorry if most of this doesn’t make much sense as of the moment. My mind is still a little fuzzy with this either. I’ll post an update with more details to come when I finish working on this.

And here’s a random happy music for you!

Have a great day!

– CodeSushi

Toon Shading

This screenshots demonstrates a rendering technique called Toon Shading. Another term for Toon Shading is Toon Lighting or Cartoon Style Lighting. The characteristics of the toon shading technique is the abrupt transition from one color shade to the next. Unlike normal lighting that has a smooth color transition.

 

 

– CodeSushi