ASP.net - objets LINQ SQL mise en cache et la sécurité des threads DataContext

We are querying database using LINQ-SQL and then storing resulting master table objects in HTTP cache. Later, the master objects are being used to query its children, using lazy loading. Here are the relevant pieces of code - I have recreated the scenario in a new proof-of-concept app:

        if (HttpRuntime.Cache["c"] == null)
        {
            LockApp.Models.DBDataContext db = new Models.DBDataContext();

            var master = db.Masters.ToList();
            HttpRuntime.Cache.Add("c", master,
                    null, DateTime.Now.AddMonths(1), 
                    TimeSpan.Zero, CacheItemPriority.Normal, null);

        }

        ViewBag.Data = (List<LockApp.Models.Master>)HttpRuntime.Cache["c"];

And here's the razor view that is iterating over master and detail objects:

    @foreach(var m in ViewBag.Data){
        @m.Id<nbsp></nbsp>
        foreach(var d in m.Details){
            @d.Id<nbsp></nbsp>
        }
        <br />
    }

It works perfectly fine and caches the data correctly. However, it fails when there are multiple requests trying to hit the web site after cache is cleared - I am testing this using JMeter, basically hitting the site with many (50) parallel threads, and then touching web.config - I immediately start seeing one of the following two errors:

Index was outside the bounds of the array

at foreach(var d in m.Details)

this error never goes away, i.e. some data gets corrupted in cache

with following stack:

  System.Collections.Generic.List`1.Add(T item) +34
   System.Data.Linq.SqlClient.SqlConnectionManager.UseConnection(IConnectionUser user) +305
   System.Data.Linq.SqlClient.SqlProvider.Execute(Expression query, QueryInfo queryInfo, IObjectReaderFactory factory, Object[] parentArgs, Object[] userArgs, ICompiledSubQuery[] subQueries, Object lastResult) +59
   System.Data.Linq.SqlClient.SqlProvider.ExecuteAll(Expression query, QueryInfo[] queryInfos, IObjectReaderFactory factory, Object[] userArguments, ICompiledSubQuery[] subQueries) +118
   System.Data.Linq.SqlClient.CompiledQuery.Execute(IProvider provider, Object[] arguments) +99
   System.Data.Linq.DeferredSourceFactory`1.ExecuteKeyQuery(Object[] keyValues) +402
   System.Data.Linq.DeferredSourceFactory`1.Execute(Object instance) +888
   System.Data.Linq.DeferredSource.GetEnumerator() +51
   System.Data.Linq.EntitySet`1.Load() +107
   System.Data.Linq.EntitySet`1.GetEnumerator() +13
   System.Data.Linq.EntitySet`1.System.Collections.IEnumerable.GetEnumerator() +4
   ASP._Page_Views_Home_Index_cshtml.Execute() in c:\Users\prc0092\Documents\Visual Studio 2012\Projects\LockApp\LockApp\Views\Home\Index.cshtml:16

or this error

ExecuteReader requires an open and available Connection. The connection's current state is open.

at the same line foreach(var d in m.Details)

this error does go away after a while if I stop hitting the site with parallel requests

with following stack

   System.Data.SqlClient.SqlConnection.GetOpenConnection(String method) +5316460
   System.Data.SqlClient.SqlConnection.ValidateConnectionForExecute(String method, SqlCommand command) +7
   System.Data.SqlClient.SqlCommand.ValidateCommand(String method, Boolean async) +155
   System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task&amp; task, Boolean asyncWrite) +82
   System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method) +53
   System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method) +134
   System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior) +41
   System.Data.Common.DbCommand.ExecuteReader() +12
   System.Data.Linq.SqlClient.SqlProvider.Execute(Expression query, QueryInfo queryInfo, IObjectReaderFactory factory, Object[] parentArgs, Object[] userArgs, ICompiledSubQuery[] subQueries, Object lastResult) +1306
   System.Data.Linq.SqlClient.SqlProvider.ExecuteAll(Expression query, QueryInfo[] queryInfos, IObjectReaderFactory factory, Object[] userArguments, ICompiledSubQuery[] subQueries) +118
   System.Data.Linq.SqlClient.CompiledQuery.Execute(IProvider provider, Object[] arguments) +99
   System.Data.Linq.DeferredSourceFactory`1.ExecuteKeyQuery(Object[] keyValues) +402
   System.Data.Linq.DeferredSourceFactory`1.Execute(Object instance) +888
   System.Data.Linq.DeferredSource.GetEnumerator() +51
   System.Data.Linq.EntitySet`1.Load() +107
   System.Data.Linq.EntitySet`1.GetEnumerator() +13
   System.Data.Linq.EntitySet`1.System.Collections.IEnumerable.GetEnumerator() +4
   ASP._Page_Views_Home_Index_cshtml.Execute() in c:\Users\prc0092\Documents\Visual Studio 2012\Projects\LockApp\LockApp\Views\Home\Index.cshtml:16

Different things I tried

Double locking

Doesn't help

    private static object ThisLock = new object();

    public ActionResult Index()
    {

        if (HttpRuntime.Cache["c"] == null)
        {
            lock (ThisLock)
            {
                if (HttpRuntime.Cache["c"] == null)
                {

Loading child data upfront

Works, but requires constant maintenance as not all children should be loaded upfront, plus see next note

DataLoadOptions dlo = new DataLoadOptions();
dlo.LoadWith<Master>(b => b.Details);
db.LoadOptions = dlo;

Locking the master object while trying to access its children

Again, requires maintenance as all initial places where child is accessed need to be found - we are struggling with this as there are different entry paths into the site

    @foreach(var m in ViewBag.Data){
        @m.Id<nbsp></nbsp>
        lock (m){
            foreach(var d in m.Details){
                @d.Id<nbsp></nbsp>
            }
        }
        <br />
    }

Switching to entity framework

This seems to still have (sometimes - much better than linq-sql) the "open connection" issue at certain number of parallel requests (50+ on core i7) - it does go away after a while as I mentioned and I haven't seen data corruption yet.

We may end up switching to EF completely as this seems to be the only viable path - assuming data corruption doesn't show up - that is to be tested on my actual project.

I am not optimistic though, as EF data context is not thread safe either, and I think the EF data objects carry their context with them. This is probably the only question that I don't have answer to yet.

Theories on why it's broken

It looks like storing LINQ-SQL object in http cache carries the data context with it. When this context is later used by multiple threads to access children, there is some type of concurrency issue that manifests itself in either temporary connectivity issue or complete data corruption of the child object. As there's no way to disconnect/reconnect the context from the LINQ object, it looks like the only suggestion is not to cache LINQ objects that need lazy-loading of their children - a substantial number of google searches I did does not seem to give you that advice, in fact sometimes it's opposite.

I have uploaded the complete project (for Visual Studio 2012 and SQL Server 2012)

https://docs.google.com/file/d/0B8CQRA9dD8POb3U5RGtCV3BMeU0/edit?usp=sharing and a simple JMeter script that will hit your local machine with parallel requests: https://docs.google.com/file/d/0B8CQRA9dD8POd1VYdGRDMEFQbEU/edit?usp=sharing

to test, start the site and run the test - then touch the web.config on the site

        if (HttpRuntime.Cache["c"] == null)
        {
            LockApp.Models.DBDataContext db = new Models.DBDataContext();

            var master = db.Masters.ToList();
            HttpRuntime.Cache.Add("c", master,
                    null, DateTime.Now.AddMonths(1), 
                    TimeSpan.Zero, CacheItemPriority.Normal, null);

        }

        ViewBag.Data = (List<LockApp.Models.Master>)HttpRuntime.Cache["c"];