R.NET failing on library(RODBC) call (and other external libraries)

Nov 19, 2014 at 9:14 PM
Hi

I'm having major issues trying to load external libraries, in particular RODBC (though out of interest we have tried on others such as ggplot2 with the same results.

I am setting paths explicitly as well, and have given IIS-USRS and IUSR write access to all R folders just to be sure. I'm running R.NET 1.5.5 and R 3.1.0

Regardless of whether i call the expression with .AsExpression() or without, I get the same error -
'r.Evaluate("library(RODBC)")' threw an exception of type 'RDotNet.ParseException'
base: {"Error in the application."}
ErrorStatement: null
Status: Null
At this stage I'm pretty desperate for resolution so any insight would really be appreciated.

thanks a mil
N

            REngine r = REngine.GetInstanceFromID("test");

            if (r == null)
            {

                
                //REngine.SetEnvironmentVariables();
                //is deprecated, current method prefers setting path explicitly so here goes
                var oldpath = System.Environment.GetEnvironmentVariable("path");
                
                RegistryKey registryKey = Registry.LocalMachine.OpenSubKey(@"SOFTWARE\R-core\R");
                string rPath = (string)registryKey.GetValue("InstallPath");
                string rVersion = (string)registryKey.GetValue("Current Version");
                registryKey.Dispose();

                var rpathbin = System.Environment.Is64BitProcess ? string.Format(@"{0}\bin\x64", rPath) : string.Format(@"{0}\bin\i386", rPath);
                var rodbcpath = System.Environment.Is64BitProcess ? string.Format(@"{0}\library\RODBC\libs\x64", rPath) : string.Format(@"{0}\library\RODBC\libs\i386", rPath);
             
                if (Directory.Exists(rpathbin) == false)
                    throw new DirectoryNotFoundException(string.Format("could not found the specified path to the directory containing r.dll: {0}", rpathbin));
                
                var newpath = string.Format("{0}{1}{2}{3}{4}", rpathbin, System.IO.Path.PathSeparator, oldpath, System.IO.Path.PathSeparator, rodbcpath);
                System.Environment.SetEnvironmentVariable("path", newpath);

                r = REngine.CreateInstance("test");
            }

            if (r.IsRunning == false)
            {
                r.Initialize();
            }
//this works fine
            r.Evaluate(string.Format("scenarioId <- {0}", scenarioID)).AsExpression();
//this doesnt
r.Evaluate("library(RODBC)");
           //this part fails r.Evaluate("library(RODBC)")' threw an exception of type           //'RDotNet.ParseException'
//    base: {"Error in the application."}
//    ErrorStatement: null
 //   Status: Null
Developer
Nov 19, 2014 at 10:18 PM
R.NET 1.5.5 is obsolete; please use the latest NuGet package 1.5.16 which is likely to have resolved your issue, and if not you will very likely get more information as to the error, which will help other users to give you useful advice.
I regret there are two parallel nuget packages sources for R.NET, a likely source of confusion, but this is outside of my control.
Nov 20, 2014 at 5:29 AM
Hi j-m,

first, thanks so much for responding, I really appreciate your input. Unfortunately, i spent many hours on 1.5.16 before trying to revert to 1.5.5.

So I've moved the code base back, and I'm back to the following error
An unhandled exception of type 'RDotNet.EvaluationException' occurred in RDotNet.dll

Additional information: Error: package or namespace load failed for 'RODBC'
It doesnt appear to be path related as i have tried explicitly setting the environment variables as above in addition to just using SetEnvironmentVariables().

It's also happening on multiple machines so doesnt appear to be caused by my R installation.

Any ideas?

thanks again
N
Nov 20, 2014 at 8:12 AM
UPDATE:

It also appears to be affecting prior projects that did work previously. Is there any chance that VS2013 IIS EXPRESS is causing this? The other projects were built in VS2010.
Nov 20, 2014 at 9:02 PM
If youx are getting an EvaluationException, its likely that something is wrong with the logic of your R code. Do you have the message provided by the EvaluationException?
Nov 21, 2014 at 8:05 AM
Hey,

The message is only a result of trying to load the RODBC library, the line in question is
r.Evaluate("library(RODBC)");
I have run all the code in question directly in R without any issue .

As an aside, we have implemented an interim workaround here by by kicking off a shell and running the script via r in command line batch mode. While this works it's clearly a very rigid solution and not particularly maintainable. This workaround isn't an option on our other projects (which have always worked) so finding some reasonable workaround is a must.

I suspect it relates to either version discrepancies in R.NET (though we have now tested on 1.5.15 and latest and both fail) or VS2013's implementation of IIS Express (although, we dont seem to have too much luck running in IIS locally either :( )

Tx
N
Nov 21, 2014 at 7:32 PM
What are you using as an apppool/user account? That package probably isn't installed in a place where R is searching. We've setup a custom R_PROFILE_USER path as part of our chef/octopus provisioning and deployment strategy..
[Environment]::SetEnvironmentVariable("R_PROFILE_USER", $profilePath, "Process")
Start-Process -FilePath $env:R_HOME\bin\i386\Rterm.exe -ArgumentList "--slave --silent --file=installPackages.R" -NoNewWindow  -RedirectStandardError installRPackages.err
Developer
Nov 21, 2014 at 10:19 PM
As skyguy94 says, this sounds like the RODBC package is not on the library search path (distinct from PATH and R_HOME). If R_HOME was incorrectly set/detected, the default packages would not even be be loaded, but they appears to be. If we intuit correctly it is very likely you can fix this issue soon.

Below is an explanation that you may well already know all about, but bear with me that's for completeness for other potential readers.

You may want to report what .libPaths() says from R console and run from your .NET app. via R.NET.

For instance if I logged with local Administrators group privilege, I get the following output. the first entry is because I set a machine-wide R_LIBS environment variable. Being admin, install.packages() would install by default in the first available library path.
> .libPaths()
[1] "c:/Rlib"                            "C:/Program Files/R/R-3.1.0/library"
I think that if I were logged in with standard user-level privilege, I'd get something like the following. install.packges() would install by default in my user specific library, and not be available to other accounts.
> .libPaths()
[1] "c:/Users/username/R/win-library/x.y"                            "c:/Rlib"                            "C:/Program Files/R/R-3.1.0/library"
A relevant section of the R FAQ should offer comprehensive options as to how to manage multiple library locations.

BTW I am not personally doing any work using R.NET from IIS, so for more IIS specific advice you should take knowledgeable advice from skyguy94 and others.

Hope this helps!

Cheers,
J-M
Nov 28, 2014 at 9:11 AM
Hi skyguy94 and J-M

Again, thanks for applying yourselves to our problem - it's much appreciated. Yes, we had suspected early on that access rights may be an issue but unfortunately this isnt the case. We are using the default app pool but have granted permissions to the relevant folders. the .libPaths() check is also something we looked at quite early on.

To be sure though, we wrote a test project which is a template MVC5 project kicking off just the RODBC load call. Its a VS2013 project and if you have the time or inclination you can access it here:

https://dl.dropboxusercontent.com/u/2469451/TestR.7z

Thanks again
N
Nov 28, 2014 at 7:34 PM
You have a very common problem with R running under IIS. If you add 'C:\Program Files\R\R-3.1.2\bin\i386' (or x64 if you are doing 64 bit) to your system path and restart Visual Studio or IIS then the RODBC package will load correctly. This is only for running under IIS and has to do with the DLL search paths under IIS while trying to find R's subordinate DLLs.
Marked as answer by untrueaxioms on 12/2/2014 at 7:06 AM
Dec 2, 2014 at 2:05 PM
skyguy94 you are a LEGEND!!!
That did it.

Thanks so much - much appreciated!
N
May 10, 2015 at 9:03 AM
Hi untrueaxioms

i am facing the same issue as well, could you explain me what have you done?
i also calling library twitteR it fail as shown here
have you solve the issue?

i did change the code to
engine.Evaluate("library(package = twitteR, verbose = TRUE, lib.loc = path) \n");
instead of
engine.Evaluate("library(twitteR) \n");
May 13, 2015 at 8:31 AM
Hey limwesley

have you already added 'C:\Program Files\R\R-3.1.2\bin\i386' (or x64 if you are doing 64 bit) to your system path as skyguy94 suggested above. That was the final change that worked for us. If yes, let me know and i'll check the other changes we implemented.
May 13, 2015 at 12:54 PM
Edited May 14, 2015 at 5:07 AM
untrueaxioms wrote:
Hey limwesley

have you already added 'C:\Program Files\R\R-3.1.2\bin\i386' (or x64 if you are doing 64 bit) to your system path as skyguy94 suggested above. That was the final change that worked for us. If yes, let me know and i'll check the other changes we implemented.
hi untrueaxioms, thanks for your reply,
i have added as saw from here and i also confirm that my package twitteR is inside the folder as you saw here
just let you know what is my system environment
windows 8.1 pro
vs ultimate 2013 with update 4,
R v3.2.0,
R.NETv1.5.22.0
rstudio 0.98.1103

here is how i run the code
REngine.SetEnvironmentVariables();
            REngine engine = REngine.GetInstance();
            engine.Initialize();
            var path2 = engine.Evaluate("search()").AsCharacter();
        engine.Evaluate("require(twitteR)");
        var path3 = (engine.Evaluate("search()")).AsCharacter();
        engine.Evaluate("library(twitteR)");
after load the path3
var path3 = (engine.Evaluate("search()")).AsCharacter();
the library twitteR still cannot loaded as well here

the error message i got after i hit the code
engine.Evaluate("require(twitteR)");
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. 

Exception Details: RDotNet.EvaluationException: Error: package or namespace load failed for 'twitteR'
hope you can provide me the solution
May 14, 2015 at 5:31 AM
hi untrueaxioms,

how was your status? i just have new update for you from my side, and it was horrible for me,
as mention previous comment i have added R_HOME and it did not work, but after i restart i get this error

below code original was working fine and it change to not working
REngine engine = REngine.GetInstance();
and i change the system variable to R_PATH and R_HOME and it did not work as well,
after i remove all the system variable and restart it able to work again

i was testing other library other than twitteR, i found that other library such as xlsx is working fine, just twitteR and RODBC did not work
as saw in this attachment i was debug the code

here is my code to run the test and get result
engine.Evaluate("require(twitteR)"); //this did not work 
        var path3 = (engine.Evaluate("search()")).AsCharacter();
        engine.Evaluate("library(xlsx)"); // this is working 
        path3 = (engine.Evaluate("search()")).AsCharacter();
        engine.Evaluate("library(RODBC)");//this did not work as well
as your information my IIS version is 8.0.1557
May 14, 2015 at 7:34 AM
i did try some other packages install from cran-r it look like some can used, but some can't
i did try Rfacebook and Rcpp and it works,
when i try to twitteR, (reinstall)
it still cant
Developer
May 14, 2015 at 9:52 AM
I could reproduce your twitteR issue, and fix it by following the advice already in this message thread.

The short version:
  • By default do not set a R_HOME in the machine environment variable.
  • Add c:\Program Files\r\R-3.2.0\bin\i386 to the machine or user's PATH env variable (or \x64 if your IIS server runs 64 bits, see details)
  • I advise you look at the solution in rdotnet-onboarding, under folder Solutions/WebApp. Credits: this material was contributed by @skyguy94
  • Install the NuGet packages (there are quite a few - couple of minutes)
The long version:

Environment:
  • Win7 64 bits
  • R 3.2.0
  • VS2013
  • IIS Express, don't know the version
Putting breakpoints into CodeController.Execute and its constructor, in the app of the onboarding guide
library(twitteR)
and using the "immediate" debug window on breakpoints
Environment.Is64BitProcess
false
Curious, as I am running a 64 bits operating system and the build is AnyCPU. Whatever, 32 bits it is.

Now if I query for the environment variables:
Environment.GetEnvironmentVariable("PATH")
"c:\\Yadi\\Yada\\Lots\\of\paths;F:\\bin\\doxygen\\bin"
Environment.GetEnvironmentVariable("R_HOME")
"C:/PROGRA~1/R/R-32~1.0"
The call to Evaluate fails indeed.

PATH should have included "c:\Program Files\r\R-3.2.0\bin\i386", but it is missing. R.NET would have appended it. Don't know/understand why IIS does that, whatever. R_HOME has been set correctly by R.NET (and the path format for R_HOME is deliberate)

To fix it it seems I then need to:
  • kill IIS Express
  • Add c:\Program Files\r\R-3.2.0\bin\i386 to the Machine or user's PATH env ver
  • AND restart VS2013 (Huh?)
    to get a fully clean slate.
Restart in debug mode, same breakpoint, immediate window:
Environment.GetEnvironmentVariable("PATH")
"c:\\Yadi\\Yada\\Lots\\of\paths;F:\\bin\\doxygen\\bin;c:\\Program Files\\r\\R-3.2.0\\bin\\i386"
Environment.GetEnvironmentVariable("R_HOME")
"C:/PROGRA~1/R/R-32~1.0"
Then things work:
library(twitteR)
tuser <- getUser('geoffjentry')
I have not set up OAuth so I get an error of course, but the package seems to work fine.
May 14, 2015 at 12:36 PM
Hi jperraud,

just let you know that it working currently, i think i have mislead regarding about the environment variables, firstly i was not sure is for "path" is not for "R_HOME" or "R_PATH"

currently, i can execute
engine.Evaluate("library(twitteR)");
thanks agian to jperraud and untrueaxioms