You are here

Merging files creates a very huge file

1 post / 0 new
wil_e
Offline
Donator
Joined: 6 years
Last seen: 4 months
Merging files creates a very huge file

Hello,

as part of a project to implement EPG into Kodi I have been trying to merge several files into a single one. This is done with the merge-xmltv file in the Misc directory and has been OK. Indeed, merging all EPG data for France, Germany, Spain and the UK, so 1,200 channels makes a file of 223 Mo. You can say that it is really big but compressed as a xz file it takes only 15 Mb space.

This file was based on the EPG data of March 25th and took 16 hours to merge.

As this data is slowly but surely becoming out-of-date I decided to update it with the EPG data grabbed during the night. I set the update parameter to "i".

The problem is that the new file is becoming more than huge. I started, as I said, with 223 Mb but I am now approaching the size of 5 Gb! And I am only at the beginning of the process (144/1202 channels)

Also, another instance of WG++ is running and working on a copy of the same original file of 223 Mb (in aseparate directory of course) but only with a timespan of 3 (so merging data until March 31st). Again, the file gets very huge : 3.4 Go for 71 channels out of 1202.

Of course I could give it another try with less channels and less days but at some time I would to put everything together. WGmulti could be a solution but I have had a formatting problem which I haven't been able to solve yet.

Has this occurred before? Is there a solution?

Thanks,

wile_e

Brought to you by Jan van Straaten

Program Development - Jan van Straaten ------- Web design - Francis De Paemeleere
Supported by: servercare.nl