Monday, December 1, 2008

Fast Persistent Data Storage Idea

There are several types of data storage technologies out there. The hard drive is cheap but slow seek speeds kills it when random access occurs, and a physical impact can literally kill it. Flash memory is faster then a hard drive and can withstand physical impacts but it has a limited lifetime do to the number of writes it can take. Random access memory like ddr3 is even faster then the flash memory and does not have lifetime issues, but is volatile meaning it looses its data when it loses power.

A while ago I thought of an idea on how to have the speed of ram but have it persist like flash. Its rather simple really, just combine on a single chip or board a equal amount of ram and flash and a capacitor to power the ram long enough to write the rams contents to the flash.

So this is how it would work:

When powering up the device (ram, flash, capacitor, and controller) would copy the contents of the flash over to the ram and charge the capacitors.

Until the capacitors are fully charged access to the drive would be denied.

When the device is accessed all reading and writing is done to the ram.

When the device looses power it copies the Rams data over to the flash. (Could add logic to only copy what has changed.)

This increases costs quite a bit but it gets rid of all the weaknesses of known storage technologies. It will be shock resistant, fast, long lasting, and its data persistent.

Porting Mono's SslStream to SilverLight

I have recently run into another problem with Microsoft’s stripping of the silverlight runtime. This time I was looking into using Sockets, because of the crippled WCF implementation. But sockets have been badly stripped. The only thing in the System.Net.Sockets namespace is the Socket class. No TcpClient no UdpClient and no NetworkStream. The System.Net.Security namespace is also gone. Meaning no SslStream.

With no SslStream and a sorely lacking implementation of WCF, silverlight is pretty insecure. Not in the fashion of a computer virus but in how FTP transmits passwords in clear text.

I’m not sure what kind of applications Microsoft expects people to write without being able send files larger then a few megabytes with WCF or to secure their Socket’s data with an SslStream. I’m starting to think Microsoft made a very bad mistake with trying to shrink the runtime so much. I bet people will either write insecure applications and blame silverlight or avoid it all together. Deep market penetration caused by a small download will mean nothing if developers don’t write programs for silverlight.

Anyways that’s enough complaining about the free framework that Microsoft spent a lot of money to write.

A rather simple yet time consuming solution to this issue is to port Mono’s implementation of .Net APIs that are needed. I’m not sure how much effort will be needed, but it should be rather simple to do. Write access to Mono’s repository would be required keep it synced with Mono so that’s not something I can do. Basically you would just sprinkle some compiler directives to exclude classes and methods that are not possible. Also add extension methods to extend silverlight classes that don’t have needed methods. Then write build scripts that compile those for silverlight. For limited subset of classes like the NetworkStream it should be relatively easy to port them to silverlight. Classes like the SslStream will require more effort because of its reliance on Certificate stores.

Friday, November 14, 2008

A Dynamic SilverLight DataGrid.

Note: this article is about how to set the values for the DataGrid. If you want to see how to add columns dynamically check out this article:

I’ve recently come across 3 very annoying limitations in Silverlight 2.0 while trying to use the data grid. The first is that the DataTable class does not exist. This is the typical way to set the ItemSource for other .Net DataGrid/GridViews. The second is that Databinding in Silverlight CANNOT do indexes or any other special kind of binding ( it only supports property bindings. So the idea of just binding to array[index] cannot be done. The third is there is no manual way to add rows to the DataGrid. You can only set ItemsSource. Its like they ca

So the only way to feed the Datagrid is to have a class collection with the properties you need. So obviously Microsoft does not want you to have dynamic datagrids. One work around is to use IConvertor ( But this only allows one way databinding and for my project it requires two way databinding. So I obviously can’t use that.

Other people have concluded the only way around this is to use code generation to create the classes with the properties they need. As cool and powerful as code generation is, I find it to be buggy, hard to debug, and generally very nasty to work with. Since I didn’t have the time or energy to write my own CodeGen solution and can’t use any 3rd party code I figured out another solution that is easier to do, works, but is potentially very slow for Grids with a large number of columns.

Basically you use a linked list for your row. This can be done because every element in a linked list is accessible via a property. The “Next” property in a link list is what makes this possible and what can make it slow. To access the nth column you just have n-1 “Next”s in the Path followed by “Value”. In the code below I am setting a column’s Binding property.

column.Binding = new Binding(CreatePath(columnIndex));

CreatePath(int columnIndex)
StringBuilder result = new StringBuilder();
(int index = 0; index < columnIndex; index++)

When creating the actual ItemsSource create a list of link list nodes and fill it with the first node from each row. In the below code the trueSource is just a sample collection that would work.

trueSource = new object[][]
new []{ "Row", "One" },
[]{ "Row", "Two" },

<LinkedListNode<object>> itemSource = new List<LinkedListNode<object>>();
(var item in trueSource)
var row = new LinkedList<object>(item);
dataGrid.ItemSource = itemSource;

There you have it. It’s very hacky, it’s potentially slow and code generation is your only other option (that I know of). Microsoft really dropped the ball on this one.

If you have yet to figure out why it can be slow here is the reason: let’s say you have 32 columns in order to bind to the last element the “Next” property has to be called 31 times to get to the node that contains its “Value”. That is 31 extra calls then a direct binding. Since data binding uses reflection that will make it even slower. But with machines being so fast as they are now I doubt you will notice the speed hit, unless you have a lot of columns and a lot of rows. If that’s the case Code generation and all its hair pulling fun is your only other option (that I know of).

Tuesday, October 14, 2008

Drop Down Buttons for WPF

The problem is the DropDownButton for WPF does not exist. And if you are like me you need one but really don’t want to have to write one. The most common scenario for drop down buttons is putting them in a toolbar. I have found a simple way to emulate the drop down button exactly without writing a line of code. Just use put a full blown Menu in.

Thursday, July 31, 2008

Another Captcha Idea (Identify/Captcha)

With the recent failures with the captcha for gmail and other major mail service providers it has became apparent that captchas need to be improved. So I pondered on this for a while and a came up with the identify/captcha idea where you have the standard captcha but you also include a background image that has items the user has to identify. Here is a mockup of what I am talking about:

The background itself being something other then whitespace would trip up the bot makers for a while but the real kicker is the “What is in the picture?” bit. Having 6 check boxes gives 64 possible answers that’s a 1/64 chance of a bot guessing it right. If the bots get the text right 1/3 of the time then that is a 1/192 chance of the bot answering correctly. Sounds like a good odds to me.

Now let’s think about how to break this system (what the bot makers would think) well they could easily create a database of pictures and correct answers. So to foil that we will need a very large collection of pictures to make the DB unmanageable and we need to distort the pictures to make any hashing attempt fail. The rest is left as an exercise for the reader.

Now all is needed is for me to convince Google to start using this.


My brother pointed out that having to have a database of images is pretty much unacceptable because image recognition can eventually defeat any distortions. So to make this idea better the image should be a 3d rendering of the objects with random textures, positions and angles and lighting making it so you can have a virtually unlimited number of images with objects in it. This would force the bot makers to actually do object recognition. This idea is more processor intensive but you will no longer have to store the Database or create it.

Thursday, July 10, 2008

Modifying collections inside a constructor.

I’m adding this to a list of no nos.

Imagine you add the object to a collection inside the constructor. Then throw an exception? Well when you get an incomplete object inside the collection.

Saturday, June 28, 2008

Silverlight is too light.

I’m trying to make a game using my physics engine and silverlight and the engine mostly works setting the compact framework symbol gets rid of most of the compiler errors. But one error that really annoyed me was that it said List.RemoveAll was missing. Talk about annoying. I use it several times in the engine so it can’t be allowed to be missing.

I’m just glad that they allow extension methods that way I can re-add the functionality they took out.

But why remove that method? I know with silverlight they are trying to trim off the fat, but this is more like cutting out the muscle.

Wednesday, June 4, 2008

The Search for the Perfect Database.

I have been searching for a database to use in a certain application for years. And none of them ever quite met all my needs until now. The program needed a database for each machine it ran on so the first requirement was that is had to be embeddable and not have a separate install. This eliminated a majority of the mainstream database systems out there. I also wanted this database to work on both Linux and Windows. I found Firebird which actually worked well for me, but had a few issues. They were the lack of good online documentation and the bug prone Ado.Net provider; which leads to my next requirement it had to have an interface with .Net. After working with firebird for a long time I finally decided to shift my database from firebird to another system. VistaDB 3 looked really appealing to me because of it being completely managed. I even switched over to it for a while, but the documentation was lacking and it ran slow (and didn’t want to spend the time fixing that.), and would have to buy it. That was when I found H2, but it had one problem it’s a Java database. In order to use it I used IKVM.Net. But I then had another problem H2 had no ADO.Net interface so I wrote one. Thus H2Sharp was born. So far I have been pleased with H2 and my only issue so far is the ~20MB class library that comes with IKVM.

I have looked at nearly every database out on the market and have tried a few of them. So here is a list of databases I considered or used.

MS SQL Express (Used):
Windows only, separate install, resource hog, not embeddable.

Windows only, lack of needed features.

Viral GPL or pay big bucks, also was too different.

SQLite (Considered):
Not Managed, bad concurrency, ADO.Net interface was buggy.

DB2 (Considered):
The free version was not embeddable and had artificial limitations and had to pay for embeddable.

Oracle (Considered):
The free version was not embeddable and had artificial limitations, separate install.

SharpHSQL (Considered):
Power loss causes corruption, did not seem to be in active development.

IBM cloudscape, Apache Derby, JavaDB (Considered):
Would have had to install Java and .Net and have it run in a separate process.
(Considered it before I found IKVM.Net and found H2 before I found IKVM.Net)

FireBird (Used):
Embeddable only on windows, lacking documentation, ADO.Net interface was buggy.

Lacking documentation, speed was slow, had to buy.

H2 (Using):
Needs IKVM.Net’s ~20MB dlls for .Net usage. Needs H2Sharp.

Wednesday, March 19, 2008

Thursday, February 21, 2008

Jade Empire: The Installer.

I recently tried to install Jade Empire. So far it has taken over 2 hours. The most annoying part of it is that a majority of the time it is doing nothing! Absolutely Zero processor usage! It’s as if the programmers decided it was a good idea to add a few random calls to wait/sleep. This makes me angry, I don’t sit down after a long day at work and say “what game do I want to watch install today?” I remember my dad once said playing games is a waste of time, so far Jade Empire has proven him right.

Update: It was the DVD drive I used that Jade Empire did not like.

Tuesday, February 19, 2008

Creating Unions in C#

Here is a Small trick to create a C/C++ like union in C#.

          You have to use the StructLayout attribute applied to the class. the StructLayout attribute needs to be passed the parameter LayoutKind.Explicit. This makes it so you can declare the exact offset in the number of Bytes each field will have. Naturally you will need to use another attribute for each of the fields to specify this offset; the name of this attribute is FieldOffset. Both of these Attributes are in the System.Runtime.InteropServices namespace.

Here is an example where 2 32bit integers will share the same memory address as a 64bit integer.

using System.Runtime.InteropServices;
struct PairID
long Id;
int LowId;
int HighId;

Decided to Create a Blog.

I have decided that I should finally make a blog, though I don’t really have time to maintain one. People around me have started to show signs that I annoy them with my ideas, so I decided to see if the internet that is comprised of complete and utter strangers will care about my random thoughts.