Best practices for organizing large python source tree

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
Post Reply
User avatar
sliptonic
Veteran
Posts: 3457
Joined: Tue Oct 25, 2011 10:46 pm
Location: Columbia, Missouri
Contact:

Best practices for organizing large python source tree

Post by sliptonic »

The Path tree is getting kind of unwieldy. We started out following the general guidelines in the wiki page.

/Mod/Path/
/App
/Gui

Most of the python ended up in the PathScripts sub-directory. Eventually as things grew, PostProcessors took on a life of its own. Then tools. Then unit tests. Then some 'common helpers'. etc. You get the idea. Now there's a lot redundancy in file names. We sometimes have problems with circular dependencies, and it's tough for new developers to ramp up.

We're starting to talk about reorganizing things to make it cleaner and avoid these problems.

I've looked around for guidance on managing large python source trees. I haven't found much. Most tutorials assume all the python is in one directory, give the same advice about naming conventions, and move on to talking about PEP8. Not terribly helpful. Anyone aware of better resources or books?

If the next version is really going to be a 1.0, I'd prefer to do this cleanup now.
heda
Veteran
Posts: 1348
Joined: Sat Dec 12, 2015 5:49 pm

Re: Best practices for organizing large python source tree

Post by heda »

+1 for the initiative (although i have zero skin in the wb as such, and have never used it)

is not a structure very dependent upon how one for example makes the classes, intra-lib dependencies, etc.
circular dependencies is a pita though...

probably the best is to just take a glance at large known projects to see what they have done
or a small write up, not for large projects...
https://docs.python-guide.org/writing/structure/

so in the end probably a matter of taste.

regardless a complete outsider and amateur view
fwiw maybe the starting point should simply be a lower case "path"

as in

Mod/Path
Mod/Path/InitGui.py
Mod/Path/path/ # this is now effectively a lib name, i.e. corresponding to a pip install "path"
Mod/Path/path/foo # foo is then your first organising level in the lib

foo could be "generators", "commands", "gui", "operations", "tools" or what is needed for the wb.

imports could then be

from path import foo
or
import path.foo as pfoo
or, if foo has function bar in it
from path.foo import bar


note that this is almost like the draft remake, although imho they should have taken it one step further and instead of making for example

Mod/Draft/draftfunctions

it could have been

Mod/Draft/draft/functions

allowing for an import like

import draft.functions as dfunc
instead of
import draftfunctions as dfunc

this of course opens up for name clashing,
but that risk is imho there today anyways,
and if one uses for example pfunc and dfunc as alias, that clash-risk is mitigated...
Post Reply