In C#, simply using a concurrent dictionary isn't enough. You still have to protect against long load times!

My large project mentioned in the earlier post uses multiple data sources. I have a server-side cache mechanism built which uses Func<> and MemoryCache to store items, with a key. If the key doesn't exist or the object is null, it will call the method to get the object, then store it.

So you call it like 

cache.GetOrStore(key, () => getItemsFromDatabase());

The main issue arose, and I should have seen this earlier, when multiple requests to the same resource try to get something from cache which takes a long time to process. This will kill the server under the right circumstances! The way around this, of course, is to use mutual exclusions. This will work like this:

- try to get item from memcache
- if exists, return the item from memcache
- if not exists, enter a mutual exclusion with the passed in key so no other requests try to run the long process
- at this point, any waiting items should check the memcache again because the first one through the gate could have populated the cache
-- if after checking the cache again, it is in there, then return it from the cache
- the first one through the gate for a specified key calls the getter and updates the cache
- the newly created item is returned

This solved the issue of crashing the server sometimes :)

Then there was another instance of server crashing. In other cases where I don't want to load the entire list of things into cache at one time, I load it on demand. So, a part of my code will request that the service find any finite number of "players" (actual athletes who compete in the sport, who have profiles). It could be a list of 100 or 1. Or any number in between.

The site uses Solr to store documents, and it is super fast. I take the list of 100 player ids, find out what is already cached, and search Solr for the rest. There could also be players who aren't in the content management system, for these players I just build a player object with what I can.

The problem is pulling the player data from the Sitecore database. It is sloooowww. It is eventually cached internally, even outside of my cache. The process was as follows:

- receive a list of player ids to look up
- get the dictionary from cache (a ConcurrentDictionary used to map player ID to the player object)
- create a new list of players to return
- for each requested player id, add to the list any that we already have cached
- for all other ids, search solr. we'll get some players back, but maybe not the remainder. Bind the solr results to Player objects and add them to the list to be returned, and to the cache
- for the remainder of the ids, search the database which will have all of the players, but not nice content management things like images and biographies.

The step which involved searching solr (which is fast), also involved binding them to Player objects, which is slow. So you can see a request coming in with 100 player ids, and that same request happening 10-20 times in a concurrently executed web server. This is where the site blows up :)

The new process is like this:

- receive list of ids.
- get dictionary from cache with id => Player mapping
- if all IDs are cached, just do a ( return ids.Select(pid => dictionary[pid]) )  --- a short circuit method to get around allocating two lists below
- check cache for player ids, add to the list, add not cached to a list of ids to search
- at this point, we enter mutual exclusivity for the entire method. the earlier approach did it per key.
- again, we check the cache for the list of players that aren't cached, and build a new list to search solr for (this is the same pattern as above. Lock, then check cache again)
- search solr for players, build the list of found players, add them to the cached dictionary.
- figure out any players we haven't found yet, get it from the players database without content management things like images, construct a Player object with what we have, and add it to the list and to the cache.

The site is much more stable now :)

Happy coding!