Issues 1665 & 1666; Symbolic links & backups

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
Sam
Posts: 177
Joined: Sun Mar 20, 2016 6:19 pm

Issues 1665 & 1666; Symbolic links & backups

Post by Sam »

TL;DR:
I propose to make FreeCAD respect symlinks, allow FreeCAD to open its own backup files, add an option like "File>Rollback..." to do that from a menu somewhere, and add an option for a backup directory.


I was snooping around #lowhangingfruit, and these got my attention. They encompass some brokenness, and touch on a minor issue I have.

issue #1666 Respect symbolic links
issue #1665 Allow setting higher value for number of backup files

I think it's within my skill set to fix this, if we can agree on what is broken and what the fix looks like.

Here's the version info that caused my opinion:

Code: Select all

OS: Ubuntu 16.04.3 LTS
Word size of OS: 64-bit
Word size of FreeCAD: 64-bit
Version: 0.17.11937 (Git)
Build type: Debug
Branch: master
Hash: 2751a3e8476c9b12d6982c7301bedd35a61fee4a
Python version: 2.7.12
Qt version: 4.8.7
Coin version: 4.0.0a
OCC version: 7.1.0
Locale: English/UnitedKingdom (en_GB)
Commentary
The description in 1666 is not perfect, and I disagree slightly:
issue #1666 wrote: 1.) Create new folder and save test .fcstd file in this folder
2.) Create symbolic link to test .fcstd file (ln -s) from different directory
2.) Open the file in FreeCAD by clicking on just created symbolic link
3.) Modify the file and save the changes
Let's do that and see what happens:
  1. Assume two directories ~/dir1 & ~/dir2
  2. Create file1.fcstd in dir1
  3. ln -s ~/dir1/file1.fcstd ~/dir2/link1.fcstd

    Code: Select all

    $ls -l dir*
    dir1:
    total 8
    -rw-rw-r-- 1 owner group 7009 Sep  1 15:43 file1.fcstd
    
    dir2:
    total 0
    lrwxrwxrwx 1 owner group 28 Sep  1 16:06 link1.fcstd -> /home/owner/dir1/file1.fcstd
    
  4. Open link1 in FreeCAD
  5. Save it

    Code: Select all

    $ ls -l dir*
    dir1:
    total 8
    -rw-rw-r-- 1 owner group 7009 Sep  1 15:43 file1.fcstd
    
    dir2:
    total 8
    -rw-rw-r-- 1 owner group 7023 Sep  1 16:09 link1.fcstd
    lrwxrwxrwx 1 owner group   28 Sep  1 16:06 link1.fcstd1 -> /home/owner/dir1/file1.fcstd
    
OK. We can see what's happening. I disagree somewhat with the analysis in the issue report:
issue #1666 wrote: Expected behaviour:

FreeCAD saves the changes at original test .fcstd file location. Backup files are saved to original test .fcstd file location.

Current behaviour:

FreeCAD saves the changes at symbolic link test .fcstd file location. Backup files are saved to symbolic link test .fcstd file location.
I think:
  • Expected behaviour:
    FreeCAD respects the link. The file is saved at the link destination, and creates a backup in the link directory.

    Code: Select all

    $ ls -l dir*
    dir1:
    total 8
    -rw-rw-r-- 1 owner group 7009 Sep  1 15:43 file1.fcstd
    
    dir2:
    total 8
    -rw-rw-r-- 1 owner group 7023 Sep  1 16:09 link1.fcstd -> /home/owner/dir1/file1.fcstd
    lrwxrwxrwx 1 owner group   28 Sep  1 16:06 link1.fcstd1
    
  • Current behaviour:
    FreeCAD moves the original file to the backup, and saves a new file in the current directory:
This seems blatantly broken to me.

In testing this, I noticed something else I hadn't expected: The most recent backup has the largest number. That is to say; .fcstd1 is older than .fcstd2; I can think of reasons it was implemented like this, but I find it odd from a user perspective. I might be alone in that though.

Also, it turns out that FreeCAD can't open its own backup files. That seems broken too. I have to rename the file to open it.

Something in issue #1665 caught my attention:
issue #1665 wrote: allow the user to not mix working file and backups at the same locations
I like this idea. Not for the reasons in the issue, but because I'd like to be able to exclude a specific folder from by normal backup routine; there is no way to de-duplicate a compressed file.

Not a big deal, and I don't want to rock that boat here, but looking at a sample of 1, it seems that the compression ratio is about 4:1. It seems likely that several MB files would be easily manageable on modern hardware. Maybe only the backup needs to be compressed? That would necessitate a file format change though. I'm thinking MIME embedding. Anyone have an opinion?

Proposal
  1. Fix the symbolic link issue such that the file is saved at the link destination.
  2. Allow FreeCAD to open its own backup files. But not necessarily set them up with their own MIME types.
  3. Add a user configuration option for backup file location that may be an absolute path, or a path relative to the current file location, eg. ~/.FreeCAD/backups or ./backups
  4. Add a menu File>Rollback... that allows picking from the available backups by time of creation.
  5. Change the order of backup file numbering such that the most recent backup is always #1
That was a lot of words, for what looks like a few simple things.

What do you hate about my ideas?
Last edited by Sam on Sun Sep 03, 2017 1:30 am, edited 1 time in total.
User avatar
DeepSOIC
Veteran
Posts: 7896
Joined: Fri Aug 29, 2014 12:45 am
Location: used to be Saint-Petersburg, Russia

Re: Issues 1665 & 1666; Symbolic links & backups

Post by DeepSOIC »

IMO symlink is not a shortcut, and the reporter was abusing symlink as a shortcut. But the expected behavior you are proposing actually makes sense, although the thing that happens now kind of makes sense too :roll: . It just reveals how exactly FreeCAD writes out files and backups - as efficient as it can possibly be.
Sam
Posts: 177
Joined: Sun Mar 20, 2016 6:19 pm

Re: Issues 1665 & 1666; Symbolic links & backups

Post by Sam »

DeepSOIC wrote: Sun Sep 03, 2017 12:54 am the thing that happens now kind of makes sense too :roll: . It just reveals how exactly FreeCAD writes out files and backups - as efficient as it can possibly be.
For certain definitions of "makes sense." ;) I do not dispute the efficiency, but IMO there is no way that secretly renaming my symlink is a useful feature. Nobody accidentally creates a link, and after you've made it you expect it to work, you know; how they work. If efficiency is real issue, and it certainly may be, it's still possible to code for this scenario.

Aside: Part of me suspects that it'd actually be slower to compress and write a file to an SSD than simply writing it. But I haven't tested. And I don't know how big an average file is.
triplus
Veteran
Posts: 9471
Joined: Mon Dec 12, 2011 4:45 pm

Re: Issues 1665 & 1666; Symbolic links & backups

Post by triplus »

Hi @Sam.

Symbolic links support would be nice to have. I don't know how you read and understood the feature request. But to me it sounds like it is aligned with what you would like to implement:
Sam wrote: Sun Sep 03, 2017 12:05 am FreeCAD respects the link.
Therefore as for the short version. FreeCAD currently lacks the ability to work with symbolic links. A lot of other applications don't.
DeepSOIC wrote: Sun Sep 03, 2017 12:54 am IMO symlink is not a shortcut, and the reporter was abusing symlink as a shortcut. But the expected behavior you are proposing actually makes sense, although the thing that happens now kind of makes sense too :roll: . It just reveals how exactly FreeCAD writes out files and backups - as efficient as it can possibly be.
No this is not about shortcuts. It's about FreeCAD currently not respecting symbolic links.

P.S. As for the ability to define optional backup file folder. Useful but likely a bit complex. As you can have files with the same name in different folders and FreeCAD would create backups in the same folder. And things like that. Trying to find solution for such scenarios can become complex quickly.
chrisb
Veteran
Posts: 53922
Joined: Tue Mar 17, 2015 9:14 am

Re: Issues 1665 & 1666; Symbolic links & backups

Post by chrisb »

While I absolutely pro points 2, 3, 4 and am neutral to point 5 (ordering of backups) I am not sure about the symbolic links. If I create a symbolic link - and it has been pointed out, that this does not happen accidentally - then it should behave like a file in the very place of it's creation. Perhaps that's exactly what I want: have a kind of temporary place where all the editing takes place while the file in fact lives somewhere else. Why should I use a symbolic link, if everything takes place in the original position?
I would rather like to see the possibility to set a backup directory for single files - in addition to the global setting.
A Sketcher Lecture with in-depth information is available in English, auf Deutsch, en français, en español.
chrisb
Veteran
Posts: 53922
Joined: Tue Mar 17, 2015 9:14 am

Re: Issues 1665 & 1666; Symbolic links & backups

Post by chrisb »

I just checked: The ordering of backup files is even worse than reversed, it is cyclic; i.e. after the last number is reached, it restarts from *.fcstd1.
A Sketcher Lecture with in-depth information is available in English, auf Deutsch, en français, en español.
Sam
Posts: 177
Joined: Sun Mar 20, 2016 6:19 pm

Re: Issues 1665 & 1666; Symbolic links & backups

Post by Sam »

chrisb wrote: Sun Sep 03, 2017 11:04 pm I am not sure about the symbolic links. If I create a symbolic link - and it has been pointed out, that this does not happen accidentally - then it should behave like a file in the very place of it's creation. Perhaps that's exactly what I want: have a kind of temporary place where all the editing takes place while the file in fact lives somewhere else. Why should I use a symbolic link, if everything takes place in the original position?
I think we agree. Let me check:
  • If you modify a link, the original file is updated.
  • Assuming the link is not in the same directory as the original file, then any ancillary stuff is created or changed at, or relative to, the link location.
Yes?

If we agree, then you might see my point that it is unexpected for FreeCAD to rename the link and therefore no longer update the original file. That is to say, it causes updates to the original file to be stored at the link location, not the original file location.
chrisb wrote: Sun Sep 03, 2017 11:06 pm I just checked: The ordering of backup files is even worse than reversed, it is cyclic; i.e. after the last number is reached, it restarts from *.fcstd1.
Gah!

I guess it's technically sound. And absolutely functional. Just not expected.
triplus
Veteran
Posts: 9471
Joined: Mon Dec 12, 2011 4:45 pm

Re: Issues 1665 & 1666; Symbolic links & backups

Post by triplus »

I did a quick test with some text editors and in addition LibreOffice on Linux. All of them did the expected thing and didn't overwrite the symbolic link on save operation. LibreOffice for example has some options to set the backup folder:

https://help.libreoffice.org/Common/Sav ... omatically

They didn't over complicate things and therefore the backup file with the same name gets overwritten. In short FreeCAD not respecting symbolic links is more or less a bug. Having a way to define backup folder is more of a feature request. ;)

P.S. I noticed text editors save backup files at the symbolic link location. I don't know what would be better from FreeCAD point of view if this area would be tackled. To save backup files at symbolic link location or at the original file location.
Sam
Posts: 177
Joined: Sun Mar 20, 2016 6:19 pm

Re: Issues 1665 & 1666; Symbolic links & backups

Post by Sam »

triplus wrote: Mon Sep 04, 2017 12:10 am I don't know what would be better from FreeCAD point of view if this area would be tackled. To save backup files at symbolic link location or at the original file location.
In my opinion it it only sane to operate at the link location. If you wanted backups at the original location, that is where you would have opened the file from.
triplus wrote: Mon Sep 04, 2017 12:10 am Having a way to define backup folder is more of a feature request. ;)
Yep. I don't know if it's desirable, but I'd use the option. Let's see how strong the opinions are.
chrisb
Veteran
Posts: 53922
Joined: Tue Mar 17, 2015 9:14 am

Re: Issues 1665 & 1666; Symbolic links & backups

Post by chrisb »

Sam wrote: Mon Sep 04, 2017 12:01 am I think we agree. Let me check:
  • If you modify a link, the original file is updated.
  • Assuming the link is not in the same directory as the original file, then any ancillary stuff is created or changed at, or relative to, the link location.
Yes?

If we agree, then you might see my point that it is unexpected for FreeCAD to rename the link and therefore no longer update the original file. That is to say, it causes updates to the original file to be stored at the link location, not the original file location.
chrisb wrote: Sun Sep 03, 2017 11:06 pm I just checked: The ordering of backup files is even worse than reversed, it is cyclic; i.e. after the last number is reached, it restarts from *.fcstd1.
Gah!

I guess it's technically sound. And absolutely functional. Just not expected.
YES to all. (One exception: I am already married). Sorry if I wasn't clear enough. The link must not be renamed, the file should be overwritten. Efficiency can be increased by moving the original file to its backup file, wherever that will be.
I agree especially on the last point as well. The usage of backup files is not everyday's business, so some peculiarities are allowed. If FreeCAD can open and roll back itself it should of course take care of that.

There is something more to be considered: symbolic links can have different names. If the backups are written to the directory of the symbolic link it is clear what should happen: The backup files get the prefix name of the link. If they are written to the files directory you have to decide again what to use.

The behaviour described in the OP is rather the behaviour of a hard link (which I have never seen in all my life, which implies that I never use them).
A Sketcher Lecture with in-depth information is available in English, auf Deutsch, en français, en español.
Post Reply