Current Project - LINQ and Dynamic Lambda

I have been working lately to redesign a system that is currently very database centric. It is an EDI system that produces both input and output files. For inbound files, we take in various file formats and map them to our business data and either insert or update our data. What we insert/update depends entirely on how our clients implemented their case. The same goes for outbound files. Each case is different.

The system was originally implemented with all the system logic in SQL Server stored procedures. We also archive all of the clients data every time we produce a file. Each new file means a new stored procedure and a new table to hold the archive data. The archive data is used to do change analysis. Many of our clients only want to see data that has changed in their files.

After several years, the system is extremely large and growing much too fast. What we need is a more flexible system where the business logic is in .Net code. We want to store only data that has changed since the last time a file was produced.

Here is what I'm thinking...

We need a database that contains metadata about our data. It will also contain information like how the data relates together. Our clients can then go in and configure what data that they want to see in the files.

We can then use LINQ and dynamic Lambda expressions to put the clients configuration together. I have not found many resources on how to build the dynamic Lambda expressions. I will be sharing some of the things I've discovered.