Sketcher performance - why is Python the culprit?
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
Be nice to others! Respect the FreeCAD code of conduct!
Re: Sketcher performance - why is Python the culprit?
UI c++ code, calls Python code, the latter calling c++ code.
This architecture decision accounts for a huge percentage of the success of FreeCAD, as effectively it enables about everything being controllable from Python, making macros and even Python WBs possible, which in terms of flexibility is difficult to match. The fact that Python, specially in the sketcher, essentially calls c++ code, should not make it inherently inefficient.
I do not use VC or Windows, so I am not sure what is actually being considered under Python in VC's profiler.
When it is necessary to add multiple geometries and constraints, it is best from a performance point of view to do it all at once:
https://github.com/FreeCAD/FreeCAD/blob ... Polygon.py
When adding each object properties needs to recalculate, the solver needs to be called and this is time, changes in properties generate calls to objects monitoring changes in properties, which involves also redrawings at UI level, that is also time. Again I do not know how c++ code called from Python is accounted for in your profiling.
That said, it would be great if you could dig out what exactly is taking the Lion's share of that time. It may be improvable.
This architecture decision accounts for a huge percentage of the success of FreeCAD, as effectively it enables about everything being controllable from Python, making macros and even Python WBs possible, which in terms of flexibility is difficult to match. The fact that Python, specially in the sketcher, essentially calls c++ code, should not make it inherently inefficient.
I do not use VC or Windows, so I am not sure what is actually being considered under Python in VC's profiler.
When it is necessary to add multiple geometries and constraints, it is best from a performance point of view to do it all at once:
https://github.com/FreeCAD/FreeCAD/blob ... Polygon.py
When adding each object properties needs to recalculate, the solver needs to be called and this is time, changes in properties generate calls to objects monitoring changes in properties, which involves also redrawings at UI level, that is also time. Again I do not know how c++ code called from Python is accounted for in your profiling.
That said, it would be great if you could dig out what exactly is taking the Lion's share of that time. It may be improvable.
-
- Posts: 34
- Joined: Tue Dec 06, 2022 1:41 am
- Location: Melbourne, Australia
- Contact:
Re: Sketcher performance - why is Python the culprit?
Don't take me wrong, I'm not contesting the benefits of scripting or the decision to base it on Python, my "why" is not challenging the things.abdullah wrote: ↑Sun Dec 18, 2022 9:28 am UI c++ code, calls Python code, the latter calling c++ code.
This architecture decision accounts for a huge percentage of the success of FreeCAD, as effectively it enables about everything being controllable from Python, making macros and even Python WBs possible, which in terms of flexibility is difficult to match.
My "why" is asking for reasons I'm seeing what I'm seeing and/or maybe someone pointing to eventual mistakes that I may be making when looking into the guts of FC. Or any piece of information that would help me get a better understanding and help me get to the answers myself.
The opening "UI c++ code, calls Python code, the latter calling c++ code." is one such piece of the puzzle, thank you for that.
That's part of the things I'm trying to understand (and, possibly, if other obligations allow) mitigate.
I mean, I started to look at the code and find things like
Code: Select all
deriv += ( (-n2.dx)*n2.y / pow(n2.length(),2) + n2.dy*n2.x / pow(n2.length(),2) )
My very first impulse was to correct that (and many others), but then I asked myself if there's gonna be of any notable benefit.
This is how my slide into looking at the performance as a whole started. (and don't ask me why I started to look at the code)
No worries, mate, I'll get to the bottom of it.abdullah wrote: ↑Sun Dec 18, 2022 9:28 am I do not use VC or Windows, so I am not sure what is actually being considered under Python in VC's profiler.
...
When adding each object properties needs to recalculate, the solver needs to be called and this is time, changes in properties generate calls to objects monitoring changes in properties, which involves also redrawings at UI level, that is also time. Again I do not know how c++ code called from Python is accounted for in your profiling.
Oh, wow. Thanks, adding list of geos and list of constraints, that's new to me, nothing that I stumbled upon suggested it as possible (the API wiki pages are a little bit... ummm... sketchy).When it is necessary to add multiple geometries and constraints, it is best from a performance point of view to do it all at once:
https://github.com/FreeCAD/FreeCAD/blob ... Polygon.py
Pinky promise, that I'll do. When I'm getting to the bottom of it, I'll probably start a new thread.That said, it would be great if you could dig out what exactly is taking the Lion's share of that time. It may be improvable.
Re: Sketcher performance - why is Python the culprit?
I did profiling in the past too and there was quite a lot of improvement in some areas thanks to it. I am happy if you identify some area of improvement, I am happy if you decide to contribute some code and I am happy if you are just interested in the guts and learn your way through this way.acolomitchi wrote: ↑Sun Dec 18, 2022 10:25 am Don't take me wrong, I'm not contesting the benefits of scripting or the decision to base it on Python, my "why" is not challenging the things.
Oh! It is always good to look at the code. You do not have to justify your answer. I did the same some years agoacolomitchi wrote: ↑Sun Dec 18, 2022 10:25 am This is how my slide into looking at the performance as a whole started. (and don't ask me why I started to look at the code)
Re: Sketcher performance - why is Python the culprit?
Sadly there is not yet a real Sketcher scripting page, like many other WB in FreeCAD.acolomitchi wrote: ↑Sun Dec 18, 2022 10:25 am ...
Oh, wow. Thanks, adding list of geos and list of constraints, that's new to me, nothing that I stumbled upon suggested it as possible (the API wiki pages are a little bit... ummm... sketchy).
...
But from my personal experience, I could say that what abdullah is saying it true, (when he speaks about the scriptability of FC).
Take in account a couple of things, that in future will improve things:
- FreeCAD is not static, and usually if something could be improved, developers are open to improvement and suggestion. (Some of them maybe seem hostile at a first glance, but usually they are not)
- Python with 3.11 version is improving his speed as lead developers have the speed improvement among their goals, so probably with future versions, (maybe even in 1.0) something will be visible on this side.
Speaking of python, some work could be done in trying to optimize the code, as python is very flexible, something is will suffice to use tuples instead of lists for "disposable" intermediate constructs, to save memory and speed as tuples are immutable.
Sadly there is no a real way of "dispose things" once used, but usually garbage collection is efficient.
Sometimes it will hang, due to error in code and sadly the only way you have is to close FreeCAD, (sadly this happens to me form time to time), but even if you use the internal editor, the first thing that FreeCAD will do when launching a script is to save the file, so your work is safe, and you could amend things before relaunching the script.
Usually the real "bottleneck" is OCCT, in the sense that what you ask OCCT to do is a computational heavy task, and FreeCAD is "single threaded" so even if you have a 16 core processor, it will use one core only (something is changing, but introducing such changes is very "delicate" and so things are slow.), something could be done and in FreeCAD some part of code have been optimized to use more thread to execute some operation in parallel, (or at least someone is working to do such changes, as this matter was discussed sometimes in developer forum).
This is not realted to Sketcher, but probably could be interesting.
I've even manage to obtain some fillets (that are very delicate things) using Python and Part WB, so I'm quite happy with it (the real Fillet operation performed by OCCT),
Strangely enough even if the surface was cuved it has worked at the first attempt, the only drawback is that I had to find the edge number using the Gui to pass it correctly in the code:
Code: Select all
obj_b2 = obj_b1.makeFillet(1.5, 1.5, [obj_b1.Edges[1]])
But probably I'm going off topic now.
Sorry for any inconvenience.
Kind Regards.
Carlo D.
GitHub page: https://github.com/onekk/freecad-doc.
- In deep articles on FreeCAD.
- Learning how to model with scripting.
- Various other stuffs.
Blog: https://okkmkblog.wordpress.com/
- In deep articles on FreeCAD.
- Learning how to model with scripting.
- Various other stuffs.
Blog: https://okkmkblog.wordpress.com/
-
- Posts: 34
- Joined: Tue Dec 06, 2022 1:41 am
- Location: Melbourne, Australia
- Contact:
Re: Sketcher performance - why is Python the culprit?
Yeah, nah, mate. Wasn't trying to stop anyone asking for justifications, rather the implied continuation would be along the line of "it's embarrassing".abdullah wrote: ↑Sun Dec 18, 2022 6:43 pmOh! It is always good to look at the code. You do not have to justify your answer. I did the same some years agoacolomitchi wrote: ↑Sun Dec 18, 2022 10:25 am This is how my slide into looking at the performance as a whole started. (and don't ask me why I started to look at the code)
Rookie mistake - spending more than a day to connect the "wire" that I obtained by making an 90+ elements array of the basic "ridge" pattern (to obtain a ridged surface by padding the resulted sketch) using the GUI. 'bout 300 elements in a sketch with zillions of coincident constraints to be manually added one by one after dragging the elements resulted from the array operation one by one in its approx place before starting to constrain it.
This is how I got into scripting (and saw that even that is not the way to go) and this is how I learned that "keep you sketches simple, make more of them if you need to" is the way of life in Part Design. Hard leaned one too, won't forget it that easy.
The best part: I do have an explanation for the impacted performance (new post pending).
-
- Posts: 34
- Joined: Tue Dec 06, 2022 1:41 am
- Location: Melbourne, Australia
- Contact:
Sketcher performance - Python is NOT the culprit
(Breaking news for newbies like me:)
Python is NOT the culprit, new data shows
Python is NOT the culprit, new data shows
Re: Sketcher performance - why is Python the culprit?
Can I make an observation here plus a question.
I have read a little on how the sketcher works and I am wondering if some of the issue is to do with
large arrays caused by complex sketches.
If there are say 100 constraints perhaps a 100x100 matrix is created ie 10 000 elements.
If there are 2 sketches of 50 constraints perhaps 2x50x50 is created ie 5 000 elements.
Could this be a reason, or a contributing factor, to the advice that we should have more
simpler sketches when possible because it keeps the number of elements down.
I am assuming that the sketcher module does not use any sparse matrix techniques - which I
studied in some depth and needed when I was writing my own structural analysis package about 35 years ago.
I would appreciate comments in this regard from Abdullah as this is his baby.
Regards
Tony Aimer
I have read a little on how the sketcher works and I am wondering if some of the issue is to do with
large arrays caused by complex sketches.
If there are say 100 constraints perhaps a 100x100 matrix is created ie 10 000 elements.
If there are 2 sketches of 50 constraints perhaps 2x50x50 is created ie 5 000 elements.
Could this be a reason, or a contributing factor, to the advice that we should have more
simpler sketches when possible because it keeps the number of elements down.
I am assuming that the sketcher module does not use any sparse matrix techniques - which I
studied in some depth and needed when I was writing my own structural analysis package about 35 years ago.
I would appreciate comments in this regard from Abdullah as this is his baby.
Regards
Tony Aimer
- adrianinsaval
- Veteran
- Posts: 5553
- Joined: Thu Apr 05, 2018 5:15 pm
Re: Sketcher performance - why is Python the culprit?
kinda, but more than reducing the total amount of elements the idea is to reduce the amount of elements that have to be solved at once, when you split your sketch only one of them can be edited at a time so only one of them will be solved at a time, so only 50 constrained DOF get solved so a 2500 element matrix vs 10000 (using your example)
Re: Sketcher performance - why is Python the culprit?
I can neither comment in Abdullah's name nor the solver side at all, but the reason to keep sketches simple is not only a technical one. From a user's point of view, simple sketches are, well, simpler and thus easier to understand and to maintain. The ancient romans knew already the principle of "divide et impera", divide and rule.
It seems that human brains works in analogy to a 2 or 3 dimensional matrix (sad to say: sometimes it is only a sparsely filled one), where problems of quadratic growth cause quadratic pain.
A Sketcher Lecture with in-depth information is available in English, auf Deutsch, en français, en español.
Re: Sketcher performance - why is Python the culprit?
It is almost a rule that a "normal" human brain could not easily retains more than 9 or 10 sequences, (You could easily see that usually 10 is used very widely, ad not only in the Bible).
another thing is that usually odd things, are most likely to be retained, some arguments that the fact that you have 2 hands with 5 fingers, it is the proof that you must not use more than 10 numbers,
The following is mostly a joke, but if someone has some scientific writings to submit as a link I will be very happy to hear some other theories.
Some people argue than you have even the foot, so French use some strange ways of composing numbers, 80 is quatrevingt, so four time twenty, so probably they were used to use feet to count, and not only hands, although I 've ever find difficult to bend my footh fingers.
No scientific discusssion have been done about the 12 used by some anglo saxons to divide things, probably they have six fingers per hands or probably use two other appendices to count, but what appendices could be bent to retain the sequences maybe ears? Even here I have always tried to bend my ears with no luck.
Sorry for the jokes if they could be inappropriate.
Regards
Carlo D.
GitHub page: https://github.com/onekk/freecad-doc.
- In deep articles on FreeCAD.
- Learning how to model with scripting.
- Various other stuffs.
Blog: https://okkmkblog.wordpress.com/
- In deep articles on FreeCAD.
- Learning how to model with scripting.
- Various other stuffs.
Blog: https://okkmkblog.wordpress.com/