Importing a large amount of data into spreadsheet will cause the program to be unresponsive.

Need help, or want to share a macro? Post here!
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
Post Reply
User avatar
freecadlzh
Posts: 138
Joined: Fri Mar 06, 2020 12:52 pm

Importing a large amount of data into spreadsheet will cause the program to be unresponsive.

Post by freecadlzh »

Image

A CSV file, 1.23Mb in size, contains 250000 calculated data. The file name is "csvfile.csv". In Freecad, I new a Spreadsheet object, and then used the importfile() function to load the file in the freecad console, and the return value was true. Then when I executed the instruction "freecad. Activedocument. Recommend()", the program would get stuck, and the title bar would show "no response".
I have tried. This file takes about a few seconds to open with software such as Excel, so I guess there is something to set when loading this file with spreadsheet in freecad?

The CSV file is too large to upload. You can only upload screenshots.
Image


Hope to get advice and help, thanks a lot.
Attachments
2.jpg
2.jpg (945.82 KiB) Viewed 859 times
1.png
1.png (460.41 KiB) Viewed 859 times
User avatar
freecadlzh
Posts: 138
Joined: Fri Mar 06, 2020 12:52 pm

Re: Importing a large amount of data into spreadsheet will cause the program to be unresponsive.

Post by freecadlzh »

freecadlzh wrote: Fri Jul 22, 2022 4:53 am Image

A CSV file, 1.23Mb in size, contains 250000 calculated data. The file name is "csvfile.csv". In Freecad, I new a Spreadsheet object, and then used the importfile() function to load the file in the freecad console, and the return value was true. Then when I executed the instruction "freecad. Activedocument. Recommend()", the program would get stuck, and the title bar would show "no response".
I have tried. This file takes about a few seconds to open with software such as Excel, so I guess there is something to set when loading this file with spreadsheet in freecad?

The CSV file is too large to upload. You can only upload screenshots.
Image


Hope to get advice and help, thanks a lot.
I find zip the csvfile, can upload, you can unzip it by 7zip. Thanks a lot.
Attachments
csvfile.zip
(83.79 KiB) Downloaded 14 times
User avatar
Roy_043
Veteran
Posts: 8450
Joined: Thu Dec 27, 2018 12:28 pm

Re: Importing a large amount of data into spreadsheet will cause the program to be unresponsive.

Post by Roy_043 »

It's a bit slow but I would not say that the program gets stuck. Importing takes ca 29 seconds on my machine.

Code: Select all

import FreeCAD as App
import time

start = time.time()

doc = App.ActiveDocument
ss = doc.addObject("Spreadsheet::Sheet", "Spreadsheet")
ss.importFile("C:/Downloads/csvfile.csv", ",")
doc.recompute()

print("Time: " + str(time.time() - start))

Code: Select all

OS: Windows 8.1 (6.3)
Word size of FreeCAD: 64-bit
Version: 0.21.29485 (Git)
Build type: Release
Branch: master
Hash: a236ca843fdd6674afb6d7ed1454fbd3b547f5ea
Python 3.8.13, Qt 5.12.9, Coin 4.0.0, Vtk 9.1.0, OCC 7.5.3
Locale: Dutch/Netherlands (nl_NL)
Installed mods: 
User avatar
freecadlzh
Posts: 138
Joined: Fri Mar 06, 2020 12:52 pm

Re: Importing a large amount of data into spreadsheet will cause the program to be unresponsive.

Post by freecadlzh »

Roy_043 wrote: Fri Jul 22, 2022 3:14 pm It's a bit slow but I would not say that the program gets stuck. Importing takes ca 29 seconds on my machine.

Code: Select all

import FreeCAD as App
import time

start = time.time()

doc = App.ActiveDocument
ss = doc.addObject("Spreadsheet::Sheet", "Spreadsheet")
ss.importFile("C:/Downloads/csvfile.csv", ",")
doc.recompute()

print("Time: " + str(time.time() - start))

Code: Select all

OS: Windows 8.1 (6.3)
Word size of FreeCAD: 64-bit
Version: 0.21.29485 (Git)
Build type: Release
Branch: master
Hash: a236ca843fdd6674afb6d7ed1454fbd3b547f5ea
Python 3.8.13, Qt 5.12.9, Coin 4.0.0, Vtk 9.1.0, OCC 7.5.3
Locale: Dutch/Netherlands (nl_NL)
Installed mods: 
Thanks for your help .
But, This is also too strange. I have tried several times on my console according to your code, and I have encountered the phenomenon of non response.

My system is Windows 10, And the freecad is version 0.19.

Image
Attachments
1.jpg
1.jpg (303.85 KiB) Viewed 752 times
User avatar
freecadlzh
Posts: 138
Joined: Fri Mar 06, 2020 12:52 pm

Re: Importing a large amount of data into spreadsheet will cause the program to be unresponsive.

Post by freecadlzh »

freecadlzh wrote: Fri Jul 22, 2022 3:51 pm
Roy_043 wrote: Fri Jul 22, 2022 3:14 pm It's a bit slow but I would not say that the program gets stuck. Importing takes ca 29 seconds on my machine.

Code: Select all

import FreeCAD as App
import time

start = time.time()

doc = App.ActiveDocument
ss = doc.addObject("Spreadsheet::Sheet", "Spreadsheet")
ss.importFile("C:/Downloads/csvfile.csv", ",")
doc.recompute()

print("Time: " + str(time.time() - start))

Code: Select all

OS: Windows 8.1 (6.3)
Word size of FreeCAD: 64-bit
Version: 0.21.29485 (Git)
Build type: Release
Branch: master
Hash: a236ca843fdd6674afb6d7ed1454fbd3b547f5ea
Python 3.8.13, Qt 5.12.9, Coin 4.0.0, Vtk 9.1.0, OCC 7.5.3
Locale: Dutch/Netherlands (nl_NL)
Installed mods: 
Thanks for your help .
But, This is also too strange. I have tried several times on my console according to your code, and I have encountered the phenomenon of non response.

My system is Windows 10, And the freecad is version 0.19.

Image
OS: Windows 10 (10.0)
Word size of OS: 64-bit
Word size of FreeCAD: 64-bit
Version: 0.19.19263 (Git)
Build type: Release
Branch: master
Hash: c4ca00a1d5b7922ef1d2ae3e820bb34ffdb64d9f
Python version: 3.7.3
Qt version: 5.12.5
Coin version: 4.0.0a
OCC version: 7.3.0
Locale: Chinese/China (zh_CN)
User avatar
onekk
Veteran
Posts: 6146
Joined: Sat Jan 17, 2015 7:48 am
Contact:

Re: Importing a large amount of data into spreadsheet will cause the program to be unresponsive.

Post by onekk »

It seems that you have not the latest 0.19 that for windows should be 0.19.4 (from the minor number it could be a 0.19.2 or 0.19.3).

Try to use a more recent version and see what happen, as 0.19 is now the "old stable" it would be safe to have last revision, it has many improvement and bug fixes over your version.

Plus signaling a bug for an old version that will not receive further updates and bug fixes is not feasible.

As the actual version is 0.20 probably it is worth trying to download a "portable version" for windows of latest stable and see if the problem is solved.

If there are problem even with 0.20 probably it worth some more investigation as maybe it could be fixed for the next 0.20 point release
Last edited by onekk on Sat Jul 23, 2022 7:29 am, edited 1 time in total.
GitHub page: https://github.com/onekk/freecad-doc.
- In deep articles on FreeCAD.
- Learning how to model with scripting.
- Various other stuffs.

Blog: https://okkmkblog.wordpress.com/
User avatar
freecadlzh
Posts: 138
Joined: Fri Mar 06, 2020 12:52 pm

Re: Importing a large amount of data into spreadsheet will cause the program to be unresponsive.

Post by freecadlzh »

onekk wrote: Fri Jul 22, 2022 4:31 pm It seems that you have not the latest 0.19 thatbfor windows should be 0.19.4 (from the minir number it could be a 0.19.2 or 0.19.3).

Try to use a more recent version and see what happen, as 0.19 is now the "old stable" it would be safe to have last revision, it has many improvement and bug fixes over your version.

Plus signaling a bug for an old version that will not receive further updates and bug fixes is not feasible.

As the actual version is 0.20 probably it is worth trying to download a "portable version" for windows of latest stable and see if the problem is solved.

If there are problem even with 0.20 probably it worth some more investigation as maybe it coulb be fixed for the next 0.20 point release
thanks for your help.As you say, in version 0.20, has no problem.
User avatar
onekk
Veteran
Posts: 6146
Joined: Sat Jan 17, 2015 7:48 am
Contact:

Re: Importing a large amount of data into spreadsheet will cause the program to be unresponsive.

Post by onekk »

Happy to hear that you solved your problem.

So you could mark the topic [Solved].

Go on your first post and edit it altering the title adding in front [Solved].

This will help future users that could have a similar problem.

Regards

Carlo D.
GitHub page: https://github.com/onekk/freecad-doc.
- In deep articles on FreeCAD.
- Learning how to model with scripting.
- Various other stuffs.

Blog: https://okkmkblog.wordpress.com/
Post Reply