[Bug?] AppImage version of FreeCad can't use AppImage of OpenScad (and possible other externals)
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
Be nice to others! Respect the FreeCAD code of conduct!
[Bug?] AppImage version of FreeCad can't use AppImage of OpenScad (and possible other externals)
In principle this is a "special" problem of the AppImage Version of Freecad.
I had already a discussion with the developer there: https://github.com/FreeCAD/FreeCAD-Bund ... 1201589944
There is a generell problem when calling an AppImage from another AppImage.
The first one adds some settings to the environment which might influence the second one.
Here especially the setting of LD_LIBRARY_PATH is problematic.
FreeCad-AppImage sets it so that the lib's from the mounted rootfs of the AppImage was used.
That's OK/needed/correct for executable's that reside "inside" the rootfs.
But it is not OK for executable's outside of rootfs.
In my case OpenSCAD could not find its lib's, because they are not inside the FreeCad-rootfs, they are inside the OpenSCAD-rootfs, but (i believe) AppRun of OpenSCAD did not set/change LD_LIBRARY_PATH because it's already set.
BTW: Using a wrapper around OpenSCAD-AppImage, that unsets LD_LIBRARY_PATH will fix this specail problem
More generally the best approach should be:
When calling an executable outside of the current rootfs, switch back to the environment as it was before AppRun of the AppImage was called.
AFAIK there is no such automatism build into the AppImage-World.
So an App that will be delivered as an AppImage, AppRun should do some extra work (saving the environment) and the Apps inside should call a Helper-function/script when they will call an external executable.
This Helper restores the environment, then calls the given executable with the given parameters.
All This might be done by two simple shell-scripts.
OK, long introduction, but I like to ensure that my intention and the use-case was clear.
The concrete case here was, that inside the WB OpenSCAD we have to supply a path to an external executable.
I believe that isn't the only situation inside FreeCad (there are too many WB's, that I don't know, each may need to call external executable's).
So even when not following my above sketched "full solution", FreeCad should "reset" (at least) LD_LIBRARY_PATH before calling an external executable.
OK, this might be some "overhead" for No-Appimage-FreeCad, but having a central Wrapper for calling external executable's is IMHO a good approach.
This will although support something that seams to be not implemented yet:
Using environment variables when specifying a path to an executable!
ex.: "$HOME/OpenSCADWrapper.sh"
Having one central wrapper only this must be adopted to support such extra functionality
what do you think ?
I had already a discussion with the developer there: https://github.com/FreeCAD/FreeCAD-Bund ... 1201589944
There is a generell problem when calling an AppImage from another AppImage.
The first one adds some settings to the environment which might influence the second one.
Here especially the setting of LD_LIBRARY_PATH is problematic.
FreeCad-AppImage sets it so that the lib's from the mounted rootfs of the AppImage was used.
That's OK/needed/correct for executable's that reside "inside" the rootfs.
But it is not OK for executable's outside of rootfs.
In my case OpenSCAD could not find its lib's, because they are not inside the FreeCad-rootfs, they are inside the OpenSCAD-rootfs, but (i believe) AppRun of OpenSCAD did not set/change LD_LIBRARY_PATH because it's already set.
BTW: Using a wrapper around OpenSCAD-AppImage, that unsets LD_LIBRARY_PATH will fix this specail problem
More generally the best approach should be:
When calling an executable outside of the current rootfs, switch back to the environment as it was before AppRun of the AppImage was called.
AFAIK there is no such automatism build into the AppImage-World.
So an App that will be delivered as an AppImage, AppRun should do some extra work (saving the environment) and the Apps inside should call a Helper-function/script when they will call an external executable.
This Helper restores the environment, then calls the given executable with the given parameters.
All This might be done by two simple shell-scripts.
OK, long introduction, but I like to ensure that my intention and the use-case was clear.
The concrete case here was, that inside the WB OpenSCAD we have to supply a path to an external executable.
I believe that isn't the only situation inside FreeCad (there are too many WB's, that I don't know, each may need to call external executable's).
So even when not following my above sketched "full solution", FreeCad should "reset" (at least) LD_LIBRARY_PATH before calling an external executable.
OK, this might be some "overhead" for No-Appimage-FreeCad, but having a central Wrapper for calling external executable's is IMHO a good approach.
This will although support something that seams to be not implemented yet:
Using environment variables when specifying a path to an executable!
ex.: "$HOME/OpenSCADWrapper.sh"
Having one central wrapper only this must be adopted to support such extra functionality
what do you think ?
- adrianinsaval
- Veteran
- Posts: 5553
- Joined: Thu Apr 05, 2018 5:15 pm
Re: [Bug?] AppImage version of FreeCad can't use AppImage of OpenScad (and possible other externals)
Note that while I have contributed to the FreeCAD-Bundle repo I'm not actually the dev behind it. Did you test changing the transfer mechanism as suggested in the report view message?
It appears there have been previous discussions about this:
https://forum.freecadweb.org/viewtopic.php?f=10&t=40848
https://github.com/FreeCAD/FreeCAD-Bundle/issues/107
https://github.com/FreeCAD/FreeCAD_Conda/issues/19
https://github.com/FreeCAD/FreeCAD-Bundle/issues/56
And there is a proposed fix: https://github.com/FreeCAD/FreeCAD/pull/7182 but doesn't look very well put together, if you could maybe review or provide a better PR it would be nice.
It appears there have been previous discussions about this:
https://forum.freecadweb.org/viewtopic.php?f=10&t=40848
https://github.com/FreeCAD/FreeCAD-Bundle/issues/107
https://github.com/FreeCAD/FreeCAD_Conda/issues/19
https://github.com/FreeCAD/FreeCAD-Bundle/issues/56
And there is a proposed fix: https://github.com/FreeCAD/FreeCAD/pull/7182 but doesn't look very well put together, if you could maybe review or provide a better PR it would be nice.
Re: [Bug?] AppImage version of FreeCad can't use AppImage of OpenScad (and possible other externals)
@adrianinsaval
First:
OpenSCAD will simply not run, because of wrong/missing lib's (.so)
About Your Links:
1 https://forum.freecadweb.org/viewtopic.php?f=10&t=40848
2 https://github.com/FreeCAD/FreeCAD-Bundle/issues/107
3 https://github.com/FreeCAD/FreeCAD_Conda/issues/19
4 https://github.com/FreeCAD/FreeCAD-Bundle/issues/56
5 https://github.com/FreeCAD/FreeCAD/pull/7182
Links 1, 2 & 5: These Links describe the same problem, but the solution given will only address WB OpenSCAD (and are still not implemented). My approach is a more general that will fix all problems when calling external executable's.
Link 3: This was a (user-)problem not setting the correct path to the OpenSCAD executable...
Link 4: Is a "request" to include OpenSCAD in the FreeCad Appimage
My intention was to point out, that: YES this is an AppImage/Snap/Flatpack specific problem, but it could not be fixed without some "help" from the main App.
At least the operation "restore environment to the one it was before AppRun was called" must be implemented inside FreeCad(-core?) whenever doing a "subprocess.Popen(*args, **kwargs)" (meaning a call of an external executable).
A simple implementation (on linux, using python) will be:
1. Save environment (part of AppRun)
At Start of Apprun do:
This will save the current environment into a file.
2. Restore environment before executing external executable (Part of FreeCad whenever a "subprocess.Popen..." is used):
If the file EnvAtFreeCadEntry does not exist do as normaly.
Otherwise use the EnvAtFreeCadEntry to restore the environment before calling the subprocess.
So using the solutions given inside the above Links, we could use:
Using this "central" function, the Implementation of a "interpretation" auf the args (e.g.: replace "$HOME" with the path to home) will be easy.
The function inside OpenSCADUtils.py can the be written as:
There is one problem open:
How to ensure that the file /tmp/EnvAtFreeCadEntry is valid (the AppImage is still running).
May be we could use the existens of the mount-point of the AppImage-rootfs as an indication...
First:
The problem has nothing to do with the 'transfer mechanism'.Did you test changing the transfer mechanism as suggested in the report view message?
OpenSCAD will simply not run, because of wrong/missing lib's (.so)
About Your Links:
1 https://forum.freecadweb.org/viewtopic.php?f=10&t=40848
2 https://github.com/FreeCAD/FreeCAD-Bundle/issues/107
3 https://github.com/FreeCAD/FreeCAD_Conda/issues/19
4 https://github.com/FreeCAD/FreeCAD-Bundle/issues/56
5 https://github.com/FreeCAD/FreeCAD/pull/7182
Links 1, 2 & 5: These Links describe the same problem, but the solution given will only address WB OpenSCAD (and are still not implemented). My approach is a more general that will fix all problems when calling external executable's.
Link 3: This was a (user-)problem not setting the correct path to the OpenSCAD executable...
Link 4: Is a "request" to include OpenSCAD in the FreeCad Appimage
My intention was to point out, that: YES this is an AppImage/Snap/Flatpack specific problem, but it could not be fixed without some "help" from the main App.
At least the operation "restore environment to the one it was before AppRun was called" must be implemented inside FreeCad(-core?) whenever doing a "subprocess.Popen(*args, **kwargs)" (meaning a call of an external executable).
A simple implementation (on linux, using python) will be:
1. Save environment (part of AppRun)
At Start of Apprun do:
Code: Select all
with open('/tmp/EnvAtFreeCadEntry', 'wb') as f:
pickle.dump(os.environ, f)
2. Restore environment before executing external executable (Part of FreeCad whenever a "subprocess.Popen..." is used):
If the file EnvAtFreeCadEntry does not exist do as normaly.
Otherwise use the EnvAtFreeCadEntry to restore the environment before calling the subprocess.
So using the solutions given inside the above Links, we could use:
Code: Select all
# This should be a "global" or "core" function
def CallExternalExecutable(*args, **kwargs):
kwargs.update({'stdout':subprocess.PIPE,'stderr':subprocess.PIPE})
if os.path.isfile('/tmp/EnvAtFreeCadEntry') #!!! and it is (still) valid !!!
with open('/tmp/EnvAtFreeCadEntry', 'rb') as f:
env = pickle.load(f)
kwargs.update({'env':env})
p = subprocess.Popen(*args, **kwargs)
stdoutd, stderrd = p.communicate()
return p, stdoutd, stderrd
The function inside OpenSCADUtils.py can the be written as:
Code: Select all
def check_output2(*args, **kwargs):
p, stdoutd, stderrd = CallExternalExecutable(*args, **kwargs)
stdoutd = stdoutd.decode("utf8")
stderrd = stderrd.decode("utf8")
if p.returncode != 0:
raise OpenSCADError('%s %s\n' % (stdoutd.strip(),stderrd.strip()))
#raise Exception,'stdout %s\n stderr%s' %(stdoutd,stderrd)
if stderrd.strip():
FreeCAD.Console.PrintWarning(stderrd+u'\n')
if stdoutd.strip():
FreeCAD.Console.PrintMessage(stdoutd+u'\n')
return stdoutd
How to ensure that the file /tmp/EnvAtFreeCadEntry is valid (the AppImage is still running).
May be we could use the existens of the mount-point of the AppImage-rootfs as an indication...
Last edited by gneiss on Thu Aug 04, 2022 7:11 pm, edited 1 time in total.
Re: [Bug?] AppImage version of FreeCad can't use AppImage of OpenScad (and possible other externals)
I think I found a good solution to the open problem mentioned in my last answer:
If we modify the code like this
At Start of AppRun (assumig it's a python script) do:
If we modify the code like this
At Start of AppRun (assumig it's a python script) do:
Code: Select all
env = os.environ.copy
env.update({'FreeCadAppRunDir':os.path.dirname(__file__)})
with open('/tmp/EnvAtFreeCadEntry', 'wb') as f:
pickle.dump(env, f)
Code: Select all
# This should be a "global" or "core" function
def CallExternalExecutable(*args, **kwargs):
kwargs.update({'stdout':subprocess.PIPE,'stderr':subprocess.PIPE})
FreeCadAppRunDir = os.environ.get('FreeCadAppRunDir')
if FreeCadAppRunDir is not None and os.path.isdir(mountPath) and os.path.isfile('/tmp/EnvAtFreeCadEntry'):
with open('/tmp/EnvAtFreeCadEntry', 'rb') as f:
env = pickle.load(f)
kwargs.update({'env':env})
p = subprocess.Popen(*args, **kwargs)
stdoutd, stderrd = p.communicate()
return p, stdoutd, stderrd
- adrianinsaval
- Veteran
- Posts: 5553
- Joined: Thu Apr 05, 2018 5:15 pm
Re: [Bug?] AppImage version of FreeCad can't use AppImage of OpenScad (and possible other externals)
The idea sounds good but using a fixed absolute path is a no go, it has to be associated to the session (you can have more than one instance of the appimage running at the same time under different envs) and use the appropriate tmp directory for the system
Re: [Bug?] AppImage version of FreeCad can't use AppImage of OpenScad (and possible other externals)
@adrianinsaval
Here is a approach that uses a unique file...
At Start of AppRun (assumig it's a python script) do:
That's a thing I just realised myselfThe idea sounds good but using a fixed absolute path is a no go
Here is a approach that uses a unique file...
At Start of AppRun (assumig it's a python script) do:
Code: Select all
f=tempfile.NamedTemporaryFile()
os.environ['FreeCadEnvFilePath']=t.name
pickle.dump(os.environ, f)
f.close()
Code: Select all
# This should be a "global" or "core" function
def CallExternalExecutable(*args, **kwargs):
kwargs.update({'stdout':subprocess.PIPE,'stderr':subprocess.PIPE})
FreeCadEnvFilePath=os.environ.get('FreeCadEnvFilePath')
if FreeCadEnvFilePath is not None and os.path.isfile(FreeCadEnvFilePath):
with open(FreeCadEnvFilePath, 'rb') as f:
env = pickle.load(f)
kwargs.update({'env':env})
p = subprocess.Popen(*args, **kwargs)
stdoutd, stderrd = p.communicate()
return p, stdoutd, stderrd
Re: [Bug?] AppImage version of FreeCad can't use AppImage of OpenScad (and possible other externals)
This seems useful enough that I wonder if there is already such a function and I (and other OpenSCAD WB devs) just didn't know it was in there. @wmayer does such a thing already exist?
Re: [Bug?] AppImage version of FreeCad can't use AppImage of OpenScad (and possible other externals)
This is a problem we actually have never encountered before AppImages, Snap packages and co. So, there is no such mechanism implemented yet.does such a thing already exist?
Re: [Bug?] AppImage version of FreeCad can't use AppImage of OpenScad (and possible other externals)
@wmayer, chennes
I am willing to implement this and supply a PR, but I need some "hints".
As far as I know the correct way is:
- create a fork of https://github.com/FreeCAD/FreeCAD (done)
- clone the fork to a local disk (done)
- make the changes, compile & Test
So far all is clear, but now I wan't to make the AppImage.
I think I need the source from https://github.com/FreeCAD/FreeCAD-Bundle
But what must I do to create an AppImage.
Next problem will be, that this will create changes in two separate repositories...
I am willing to implement this and supply a PR, but I need some "hints".
As far as I know the correct way is:
- create a fork of https://github.com/FreeCAD/FreeCAD (done)
- clone the fork to a local disk (done)
- make the changes, compile & Test
So far all is clear, but now I wan't to make the AppImage.
I think I need the source from https://github.com/FreeCAD/FreeCAD-Bundle
But what must I do to create an AppImage.
Next problem will be, that this will create changes in two separate repositories...
- adrianinsaval
- Veteran
- Posts: 5553
- Joined: Thu Apr 05, 2018 5:15 pm
Re: [Bug?] AppImage version of FreeCad can't use AppImage of OpenScad (and possible other externals)
@looo would be the person to ask on what to do to make an appimage, I think you need to compile freecad from within a conda environment and somehow package that.