[sage-devel] Jupyterhub kernel and SAGE_ROOT

[Jori Mäntysalo]

I am trying to marry SageMath and Jupyterhub. I think I got them engaged,
but the wedding night has a problem:

Error: You must set the SAGE_ROOT environment variable or run this
script from the SAGE_ROOT or SAGE_ROOT/local/bin/ directory.
Error setting environment variables by sourcing
'/home/jupkernelit/sage-8.4/local/bin/sage-env';
possibly contact sage-devel (see
http://groups.google.com/group/sage-devel)

First I think that I just set SAGE_ROOT in the command line before
jupytehub-command, set it with export-command, or put it to /etc/profile.
They all failed, so jupyterhub seems to ignore environment.

What next?

[François Bissey]

Didn’t we have that conversation a couple of months ago?
The message here comes from this section of code in sage-env

# New value for SAGE_ROOT: either SAGE_ROOT (if given)
# or a guessed value based on pwd.
if [ -n "$SAGE_ROOT" ]; then
NEW_SAGE_ROOT="$SAGE_ROOT"
elif [ -f sage -a -d build ]; then
NEW_SAGE_ROOT="."
elif [ -f ../../sage -a -d ../../build ]; then
NEW_SAGE_ROOT="../.."
else
# No idea what SAGE_ROOT should be...
echo >&2 "Error: You must set the SAGE_ROOT environment variable or run this"
echo >&2 "script from the SAGE_ROOT or SAGE_ROOT/local/bin/ directory."
return 1
fi

There is some more code in there to make you trip. Touching sage-env could solve your problem.
Another interesting file you could try to seed with the value of SAGE_ROOT is
/etc/environment

François

Post by Jori Mäntysalo
Error: You must set the SAGE_ROOT environment variable or run this
script from the SAGE_ROOT or SAGE_ROOT/local/bin/ directory.
Error setting environment variables by sourcing '/home/jupkernelit/sage-8.4/local/bin/sage-env';
possibly contact sage-devel (see http://groups.google.com/group/sage-devel)
First I think that I just set SAGE_ROOT in the command line before jupytehub-command, set it with export-command, or put it to /etc/profile. They all failed, so jupyterhub seems to ignore environment.
What next?
--
Jori Mäntysalo

[Jori Mäntysalo]

Didn’t we have that conversation a couple of months ago?
The message here comes from this section of code in sage-env
else
# No idea what SAGE_ROOT should be...
echo >&2 "Error: You must set the SAGE_ROOT environment variable or run this"
echo >&2 "script from the SAGE_ROOT or SAGE_ROOT/local/bin/ directory."
return 1

OK, so I just hardcoded this by commenting out return 1 and putting

NEW_SAGE_ROOT=/home/jupkernelit/sage-8.4

instead. Now it at works, including graphich. Sorry for the noise.

Now it's time for polyamory, i.e. getting also shibboleth ready. But that
will be off-topic for this list.

[Dima Pasechnik]

Post by Jori Mäntysalo

Post by François Bissey
Didn’t we have that conversation a couple of months ago?
The message here comes from this section of code in sage-env
else
# No idea what SAGE_ROOT should be...
echo >&2 "Error: You must set the SAGE_ROOT environment variable or run this"
echo >&2 "script from the SAGE_ROOT or SAGE_ROOT/local/bin/ directory."
return 1

OK, so I just hardcoded this by commenting out return 1 and putting
NEW_SAGE_ROOT=/home/jupkernelit/sage-8.4
instead. Now it at works, including graphich. Sorry for the noise.
Now it's time for polyamory, i.e. getting also shibboleth ready. But that
will be off-topic for this list.

there are jupyter discussion boards on https://discourse.jupyter.org/,
by the way.

Post by Jori Mäntysalo
--
Jori Mäntysalo

[François Bissey]

I’d be curious about that, but yes that may be off-list.

François

Now it's time for polyamory, i.e. getting also shibboleth ready. But that will be off-topic for this list.

[Jori Mäntysalo]

I’d be curious about that, but yes that may be off-list.

Of course if I can make a good step-by-step manual, then I will write
about that in this list.

[Enrique Artal]

For me it works adding "env": { "SAGE_ROOT": "where_SAGE_ROOT_is" } in
kernel.json

Didn’t we have that conversation a couple of months ago?
The message here comes from this section of code in sage-env
# New value for SAGE_ROOT: either SAGE_ROOT (if given)
# or a guessed value based on pwd.
if [ -n "$SAGE_ROOT" ]; then
NEW_SAGE_ROOT="$SAGE_ROOT"
elif [ -f sage -a -d build ]; then
NEW_SAGE_ROOT="."
elif [ -f ../../sage -a -d ../../build ]; then
NEW_SAGE_ROOT="../.."
else
# No idea what SAGE_ROOT should be...
echo >&2 "Error: You must set the SAGE_ROOT environment variable or run this"
echo >&2 "script from the SAGE_ROOT or SAGE_ROOT/local/bin/ directory."
return 1
fi
There is some more code in there to make you trip. Touching sage-env could
solve your problem.
Another interesting file you could try to seed with the value of SAGE_ROOT is
/etc/environment
François

Post by Jori Mäntysalo
I am trying to marry SageMath and Jupyterhub. I think I got them
Error: You must set the SAGE_ROOT environment variable or run this
script from the SAGE_ROOT or SAGE_ROOT/local/bin/ directory.
Error setting environment variables by sourcing

'/home/jupkernelit/sage-8.4/local/bin/sage-env';

Post by Jori Mäntysalo
possibly contact sage-devel (see

http://groups.google.com/group/sage-devel)

Post by Jori Mäntysalo
First I think that I just set SAGE_ROOT in the command line before

jupytehub-command, set it with export-command, or put it to /etc/profile.
They all failed, so jupyterhub seems to ignore environment.

Post by Jori Mäntysalo
What next?
--
Jori MÀntysalo

[Jori Mäntysalo]

For me it works adding "env": { "SAGE_ROOT": "where_SAGE_ROOT_is" } in
kernel.json

Thanks, seems to be much better solution.

Now, what is the best practise for upgrading Sage? Jupyterhub makes it
easy to have several kernels installed, but there is no real "big" and
"small" releases of SageMath. Hence it makes no sense to have "Sage 8.x
-serie" and "Sage 9.x -serie" in parallel. Can we have something like
"Sage" having newest as default, and explicit versions available for
advanced users?

With SageNB I have just installed new version. If something brokes for
some user, then it brokes -- there are deprecation warnings that hopefully
helps.

[Kwankyu Lee]

The

Post by Jori Mäntysalo

Post by Enrique Artal
For me it works adding "env": { "SAGE_ROOT": "where_SAGE_ROOT_is" } in
kernel.json

Thanks, seems to be much better solution.

A problem with this is that it is overwritten when Sage is rebuilt for
upgrade. A much much better solution is to include the following
into jupyterhub_config.py:

c.Spawner.environment = {'SAGE_ROOT': 'where_SAGE_ROOT_is'}

Good luck.

[Enrique Artal]

That's true. I just edit the kernel.json for each new release (I read it in
some tutorial for jupyterhub+sagemath). I will try this.

Post by Kwankyu Lee
The

Post by Jori Mäntysalo

Post by Enrique Artal
For me it works adding "env": { "SAGE_ROOT": "where_SAGE_ROOT_is" } in
kernel.json

Thanks, seems to be much better solution.

A problem with this is that it is overwritten when Sage is rebuilt for
upgrade. A much much better solution is to include the following
c.Spawner.environment = {'SAGE_ROOT': 'where_SAGE_ROOT_is'}
Good luck.

[Enrique Artal]

I tried your solution, it didn't work for me. In jupyter the kernel did not
start, in jupyterlab there was also some style issues.

Post by Enrique Artal
That's true. I just edit the kernel.json for each new release (I read it
in some tutorial for jupyterhub+sagemath). I will try this.

Post by Kwankyu Lee
The

Post by Jori Mäntysalo

Post by Enrique Artal
For me it works adding "env": { "SAGE_ROOT": "where_SAGE_ROOT_is" } in
kernel.json

Thanks, seems to be much better solution.

A problem with this is that it is overwritten when Sage is rebuilt for
upgrade. A much much better solution is to include the following
c.Spawner.environment = {'SAGE_ROOT': 'where_SAGE_ROOT_is'}
Good luck.

[Luca De Feo]

Post by Jori Mäntysalo
I am trying to marry SageMath and Jupyterhub. I think I got them engaged,

This is not a Sage + JupyterHub problem as much as a Sage + system Jupyter one.

Packagers have already solved it for various distributions, so, if you
don't need a Sage version compiled from sources, you can have a
working Sage + JupyterHub setup using the system packages with no
further hacks.

Here is an example of Dockerfile based on Ubuntu 18.04 (bionic) that
is currently working for me with DockerSpawner :

https://github.com/defeo/jupyterhub-docker/blob/keats/jupyterlab/Dockerfile

I have a blog post on the whole process of setting up JupyterHub using
Docker Compose (spoiler: it's easy):

https://opendreamkit.org/2018/10/17/jupyterhub-docker/

But I haven't advertised much yet because I'm still fiddling with the
contents of single-user server Docker image (spoiler: it's a mess if
you need anything else than Sage).

Hope this helps. I'd be glad to get any feedback.

Cheers,
Luca

[Jori Mäntysalo]

Post by Luca De Feo
Packagers have already solved it for various distributions, so, if you
don't need a Sage version compiled from sources, you can have a
working Sage + JupyterHub setup using the system packages with no
further hacks.

How fast will new SageMath versions be packaged? I suppose that fast
enought, and those needing the bleeding edge version can compile it
themself.

Post by Luca De Feo
I have a blog post on the whole process of setting up JupyterHub using
https://opendreamkit.org/2018/10/17/jupyterhub-docker/

Seems interesting. Here we will have about 35,000 theoretically possible
users, and maybe 10 of them will be heavy duty users. Then maybe 500 may
be users in some random course, or checking just one computation etc.

I have no previous experience on containers, but I guess they are here to
stay and I should learn them.

[Luca De Feo]

Post by Jori Mäntysalo
How fast will new SageMath versions be packaged? I suppose that fast
enought, and those needing the bleeding edge version can compile it
themself.

I agree. Ubuntu bionic has 8.1, cosmic has 8.3.

Unless we screw up and make Sage impossible to package (see mail by
Samuel on Debian freeze), these will follow closely enough for the
average user.

Post by Jori Mäntysalo
Seems interesting. Here we will have about 35,000 theoretically possible
users,

That's a lot for a single server! Unless the average use is a single
day a year :)

Post by Jori Mäntysalo
and maybe 10 of them will be heavy duty users. Then maybe 500 may
be users in some random course, or checking just one computation etc.

500 hundred simultaneous connections may also be a bit too much,
depending on how powerful your server is.

Post by Jori Mäntysalo
I have no previous experience on containers, but I guess they are here to
stay and I should learn them.

I got to the point where I consider Docker as fundamental a tool as Git.

Luca

[Jori Mäntysalo]

Post by Luca De Feo

Post by Jori Mäntysalo
and maybe 10 of them will be heavy duty users. Then maybe 500 may
be users in some random course, or checking just one computation etc.

500 hundred simultaneous connections may also be a bit too much,
depending on how powerful your server is.

I guess that 50 simultaneous connection will be the maximum. The use might
be, for example, a course on graph theory having mandatory but trivial
exercise about SageMath graph functions.

[Luca De Feo]

Post by Jori Mäntysalo
I guess that 50 simultaneous connection will be the maximum. The use might
be, for example, a course on graph theory having mandatory but trivial
exercise about SageMath graph functions.

That's a reasonable load.

[Erik Bray]

Post by Luca De Feo

Post by Jori Mäntysalo
How fast will new SageMath versions be packaged? I suppose that fast
enought, and those needing the bleeding edge version can compile it
themself.

I agree. Ubuntu bionic has 8.1, cosmic has 8.3.
Unless we screw up and make Sage impossible to package (see mail by
Samuel on Debian freeze), these will follow closely enough for the
average user.

That's funny wording. I would consider Sage effectively impossible to
package. Sure, it's been *done*. We've been doing it. But it
requires an almost absurd amount of overhead to get right :)

Post by Luca De Feo

Post by Jori Mäntysalo
I have no previous experience on containers, but I guess they are here to
stay and I should learn them.

I got to the point where I consider Docker as fundamental a tool as Git.

I wouldn't go quite that far, but there's still hardly a day goes by I
don't use one :)

[Erik Bray]

Post by Jori Mäntysalo
I am trying to marry SageMath and Jupyterhub. I think I got them engaged,
Error: You must set the SAGE_ROOT environment variable or run this
script from the SAGE_ROOT or SAGE_ROOT/local/bin/ directory.
Error setting environment variables by sourcing
'/home/jupkernelit/sage-8.4/local/bin/sage-env';
possibly contact sage-devel (see
http://groups.google.com/group/sage-devel)
First I think that I just set SAGE_ROOT in the command line before
jupytehub-command, set it with export-command, or put it to /etc/profile.
They all failed, so jupyterhub seems to ignore environment.
What next?

Perhaps only loosely related, and not immediately helpful to your
problem, but something I've been thinking about for a while: I think
it's rather unfortunate that Sage-the-library (aka sagelib) requires
certain environment variables to be set in order to work at all. I
think sagelib--specifically the sage.env module--should try to
automatically set `SAGE_ROOT` and other variables that derive from it
if they aren't already set.

Currently sagelib will just crash at import time with an unhelpful
traceback if at least SAGE_ROOT and SAGE_LOCAL are not set. Honestly,
in most cases, it can make some reasonable guesses about this. I also
thought we might be able to include an optional config file for many
of these variables which might be searched for in a few common
locations. The first place to look might be installed alongside the
sage package itself. System packagers could include their own copy of
this file in the package so that all the key variables (e.g.
SAGE_ROOT, SAGE_SRC, SAGE_DOC, etc.) are set appropriately for however
Sage is packaged on that system. IIRC Numpy does something a bit
similar, but for different reasons.

[Timo Kaufmann]

I've had similar thoughts for a while. Replacing sage-env with a dumb
config file that can be parsed from within python would go a long way. I'm
not too familiar with the native sage-env however, it might do some clever
stuff that a dumb config file couldn't do.

Post by Jori Mäntysalo

Post by Jori Mäntysalo
I am trying to marry SageMath and Jupyterhub. I think I got them

engaged,

Post by Jori Mäntysalo
Error: You must set the SAGE_ROOT environment variable or run this
script from the SAGE_ROOT or SAGE_ROOT/local/bin/ directory.
Error setting environment variables by sourcing
'/home/jupkernelit/sage-8.4/local/bin/sage-env';
possibly contact sage-devel (see
http://groups.google.com/group/sage-devel)
First I think that I just set SAGE_ROOT in the command line before
jupytehub-command, set it with export-command, or put it to

/etc/profile.

Post by Jori Mäntysalo
They all failed, so jupyterhub seems to ignore environment.
What next?

Perhaps only loosely related, and not immediately helpful to your
problem, but something I've been thinking about for a while: I think
it's rather unfortunate that Sage-the-library (aka sagelib) requires
certain environment variables to be set in order to work at all. I
think sagelib--specifically the sage.env module--should try to
automatically set `SAGE_ROOT` and other variables that derive from it
if they aren't already set.
Currently sagelib will just crash at import time with an unhelpful
traceback if at least SAGE_ROOT and SAGE_LOCAL are not set. Honestly,
in most cases, it can make some reasonable guesses about this. I also
thought we might be able to include an optional config file for many
of these variables which might be searched for in a few common
locations. The first place to look might be installed alongside the
sage package itself. System packagers could include their own copy of
this file in the package so that all the key variables (e.g.
SAGE_ROOT, SAGE_SRC, SAGE_DOC, etc.) are set appropriately for however
Sage is packaged on that system. IIRC Numpy does something a bit
similar, but for different reasons.

[Erik Bray]

I've had similar thoughts for a while. Replacing sage-env with a dumb config file that can be parsed from within python would go a long way. I'm not too familiar with the native sage-env however, it might do some clever stuff that a dumb config file couldn't do.

I'm not necessary suggesting a full replacement: Just a dumb config in
addition to it that can be used both from sage.env and perhaps
sage-env as well. I tossed about some ideas for this in
https://trac.sagemath.org/ticket/22652 and possibly elsewhere, but
never really made progress on it. I think the one "new" point I'm
making here is that `import sage` should *just work* without having to
set any special environment variables :/

Post by Erik Bray

Post by Jori Mäntysalo
I am trying to marry SageMath and Jupyterhub. I think I got them engaged,
Error: You must set the SAGE_ROOT environment variable or run this
script from the SAGE_ROOT or SAGE_ROOT/local/bin/ directory.
Error setting environment variables by sourcing
'/home/jupkernelit/sage-8.4/local/bin/sage-env';
possibly contact sage-devel (see
http://groups.google.com/group/sage-devel)
First I think that I just set SAGE_ROOT in the command line before
jupytehub-command, set it with export-command, or put it to /etc/profile.
They all failed, so jupyterhub seems to ignore environment.
What next?

Perhaps only loosely related, and not immediately helpful to your
problem, but something I've been thinking about for a while: I think
it's rather unfortunate that Sage-the-library (aka sagelib) requires
certain environment variables to be set in order to work at all. I
think sagelib--specifically the sage.env module--should try to
automatically set `SAGE_ROOT` and other variables that derive from it
if they aren't already set.
Currently sagelib will just crash at import time with an unhelpful
traceback if at least SAGE_ROOT and SAGE_LOCAL are not set. Honestly,
in most cases, it can make some reasonable guesses about this. I also
thought we might be able to include an optional config file for many
of these variables which might be searched for in a few common
locations. The first place to look might be installed alongside the
sage package itself. System packagers could include their own copy of
this file in the package so that all the key variables (e.g.
SAGE_ROOT, SAGE_SRC, SAGE_DOC, etc.) are set appropriately for however
Sage is packaged on that system. IIRC Numpy does something a bit
similar, but for different reasons.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[Timo Kaufmann]

Post by Erik Bray
I think the one "new" point I'm

making here is that `import sage` should *just work* without having to
set any special environment variables :/

I very much agree

Post by Erik Bray

Post by Timo Kaufmann
I've had similar thoughts for a while. Replacing sage-env with a dumb

config file that can be parsed from within python would go a long way. I'm
not too familiar with the native sage-env however, it might do some clever
stuff that a dumb config file couldn't do.
I'm not necessary suggesting a full replacement: Just a dumb config in
addition to it that can be used both from sage.env and perhaps
sage-env as well. I tossed about some ideas for this in
https://trac.sagemath.org/ticket/22652 and possibly elsewhere, but
never really made progress on it. I think the one "new" point I'm
making here is that `import sage` should *just work* without having to
set any special environment variables :/

Post by Timo Kaufmann

Post by Erik Bray

Post by Jori Mäntysalo
I am trying to marry SageMath and Jupyterhub. I think I got them

engaged,

Post by Timo Kaufmann

Post by Erik Bray

Post by Jori Mäntysalo
Error: You must set the SAGE_ROOT environment variable or run this
script from the SAGE_ROOT or SAGE_ROOT/local/bin/ directory.
Error setting environment variables by sourcing
'/home/jupkernelit/sage-8.4/local/bin/sage-env';
possibly contact sage-devel (see
http://groups.google.com/group/sage-devel)
First I think that I just set SAGE_ROOT in the command line before
jupytehub-command, set it with export-command, or put it to

/etc/profile.

Post by Timo Kaufmann

Post by Erik Bray

Post by Jori Mäntysalo
They all failed, so jupyterhub seems to ignore environment.
What next?

Perhaps only loosely related, and not immediately helpful to your
problem, but something I've been thinking about for a while: I think
it's rather unfortunate that Sage-the-library (aka sagelib) requires
certain environment variables to be set in order to work at all. I
think sagelib--specifically the sage.env module--should try to
automatically set `SAGE_ROOT` and other variables that derive from it
if they aren't already set.
Currently sagelib will just crash at import time with an unhelpful
traceback if at least SAGE_ROOT and SAGE_LOCAL are not set. Honestly,
in most cases, it can make some reasonable guesses about this. I also
thought we might be able to include an optional config file for many
of these variables which might be searched for in a few common
locations. The first place to look might be installed alongside the
sage package itself. System packagers could include their own copy of
this file in the package so that all the key variables (e.g.
SAGE_ROOT, SAGE_SRC, SAGE_DOC, etc.) are set appropriately for however
Sage is packaged on that system. IIRC Numpy does something a bit
similar, but for different reasons.

--
You received this message because you are subscribed to the Google

Groups "sage-devel" group.

Post by Timo Kaufmann
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "sage-devel" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/sage-devel/RuWNK52yGYg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[François Bissey]

Post by Erik Bray
I'm not necessary suggesting a full replacement: Just a dumb config in
addition to it that can be used both from sage.env and perhaps
sage-env as well. I tossed about some ideas for this in
https://trac.sagemath.org/ticket/22652 and possibly elsewhere, but
never really made progress on it. I think the one "new" point I'm
making here is that `import sage` should *just work* without having to
set any special environment variables :/

It does work in sage-on-gentoo, although it requires env.py to be properly
setup at install time. I, and now other distro maintainers, have made a point
to eliminate SAGE_ROOT as much as possible from sage runtime code. In the
python code, it is mostly gone apart from some doctests. SAGE_LOCAL is another story,
however we probably could figure it out from the python configuration rather
than a seed.

Sage’s scripts on the other hand rely heavily on sage-env and from being
called from the root sage or the sage environment (sage -sh). That may be harder
to fix.

François

[Jeroen Demeyer]

Post by François Bissey
SAGE_LOCAL is another story,
however we probably could figure it out from the python configuration rather
than a seed.

Isn't SAGE_LOCAL == sys.prefix

[François Bissey]

Post by Jeroen Demeyer

Post by François Bissey
SAGE_LOCAL is another story,
however we probably could figure it out from the python configuration rather
than a seed.

Isn't SAGE_LOCAL == sys.prefix

Should be. But following this thread I changed the way I patch
env.py and now I have
_add_variable_or_fallback('SAGE_LOCAL', sysconfig.get_config_var("prefix”))

François

[julien.puydt via sage-devel]

<div dir='auto'><div>Hi,<br><div class="gmail_extra"><br><div class="gmail_quote">Le&nbsp;6 nov. 2018 13:57, Timo Kaufmann &lt;***@gmail.com&gt; a écrit&nbsp;:<br type="attribution"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">&gt; I think the one "new" point I'm<br>
making here is that `import sage` should *just work* without having to<br><div>
set any special environment variables :/</div><div><br></div><div>I very much agree</div></div></blockquote></div></div></div><div dir="auto"><br></div><div dir="auto">More precisely : it should be possible to tweak the environment so sage can run from a snapshot installed in homedir, but by default (system-wide install), it shouldn't need anything.</div><div dir="auto"><br></div><div dir="auto">JP</div><div dir="auto"></div></div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &quot;sage-devel&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:sage-devel+***@googlegroups.com">sage-devel+***@googlegroups.com</a>.<br />
To post to this group, send email to <a href="mailto:sage-***@googlegroups.com">sage-***@googlegroups.com</a>.<br />
Visit this group at <a href="https://groups.google.com/group/sage-devel">https://groups.google.com/group/sage-devel</a>.<br />
For more options, visit <a href="https://groups.google.com/d/optout">https://groups.google.com/d/optout</a>.<br />