Memory leaks

Aug 14, 2013 at 9:17 PM
Edited Aug 14, 2013 at 9:18 PM
Hello, guys. I work with R.NET library + .NET 4.0 + MSVS 2012 (Ult). I created Excel workbook project (for comfortable data visualization). After running code used lambdas and function calls I have great memory leaks (my code breaks after using >2 gb memory by stack overflow exception).
        private void button9_Click(object sender, EventArgs e)
        {
            int windowSize = 13;
            int sheetShiftPosition = 3;
            int i = sheetShiftPosition + windowSize;
            int OHLCBarsShift = 4;
            double TRADE_PROBABILITY = 0.85;
            double totalProfit = 0.0;

            List<List<double>> OHLC = new List<List<double>>(4);
            List<List<bool>> OHLCURTest = new List<List<bool>>(4);

            for (int j = 0; j < 4; j++)
                OHLC.Add(new List<double>(windowSize));

            List<List<double>> models = new List<List<double>>();

            while (myWorksheet.Cells[i, 7].value != null)
            {
                OHLC.ForEach(x => x.Clear());

                OHLCURTest.Clear();
                models.Clear();

                int j = OHLCBarsShift;
                OHLC.ForEach(x =>
                    {
                        for (int k = 0; k < windowSize; k++)
                            x.Add(myWorksheet.Cells[i - windowSize + k + 1, j].value);
                        
                        j++;
                    });
                
                foreach (var elem in OHLC)
                    OHLCURTest.Add(GetUnitRootTest(elem));

                j = 0;
                OHLCURTest.ForEach(chkModel =>
                    {
                        if (chkModel.All(x => x == true))
                            models.Add(GetDSModelParams(OHLC[j]));
                        else
                            models.Add(GetTSModelParams(OHLC[j]));
                        
                        j++;
                    });


                var closeProb = models[3].Last();
                var lowProb = models[2].Last();
                var highProb = models[1].Last();

                myWorksheet.Cells[i, 17].value = models[3].ElementAt(1);
                myWorksheet.Cells[i, 18].value = models[3].ElementAt(2);
                myWorksheet.Cells[i, 19].value = models[3].ElementAt(3);
                myWorksheet.Cells[i, 20].value = models[3].ElementAt(4);
                myWorksheet.Cells[i, 21].value = models[3].ElementAt(5);
                myWorksheet.Cells[i, 22].value = models[3].ElementAt(0) == 0 ? "TS series" : "DS series";
                myWorksheet.Cells[i, 23].value = models[3].ElementAt(6);

                i++;
            }

        }
methods I used in lambdas and generics:
private List<double> GetDSModelParams(List<double> data);
private List<double> GetTSModelParams(List<double> data);
private List<bool> GetUnitRootTest(List<double> data);
What's my mistake? What should I do?
The code
engine.Evaluate("gc()")
doesn't help...
Coordinator
Aug 15, 2013 at 12:36 AM
I think I fix it at changeset 8cd79c172cbe, but it is not included in the latest release.
Wait for the next release, or try the latest source.
Aug 15, 2013 at 7:50 AM
But, What's the reason of these memory leaks? If I'm not mistaken, it's connected with R core and R memory management?
Coordinator
Aug 15, 2013 at 8:31 AM
It was the bug of R.NET. Runtime must prevent R from disposing an object bound to SymbolicExpression. R.NET fails to release such an object even after the object is unreferenced because of misimplementation.

Bug in detail:
Objects should not be disposed in R if it is referenced by .NET code. SymbolicExpression calls protect function to prevent the garbage collector from disposing the object in R. Whenever the object is unreferenced by .NET, it should be unprotected in R with calling unprotect function for the GC to collect it. The bug was simply the unprotect function is not called because condition determining whether the object should call the unprotect function was wrong.

So, after the bug fix, REngine.ForceGarbageCollection should work. Also, the bug tells that .NET's GC should work before R's GC. The following code may do well:
GC.Collect();
engine.ForceGarbageCollection();