Ambiera ForumDiscussions, Help and Support. |
|
|
|||||
|
It is possible to have large file > 5 GB to be read? I have tried it, but the programs ends and nothing happens? The file is xml based, larger than 5 GB in size, and has around 200 M lines. |
||||
|
Difficult to say, it is open source, just debug it. But if it doesn't work out of the box, it probably doesn't support this, no. |
||||
|
Yeah, it is open source and I have checked it already. It is using fopen to open the file. So, I have to find away to overcome this (either use ifstream or by defining define _FILE_OFFSET_BITS 64, or what ever else). I will try to post here if I managed to fix it. Thanks. Ali Awadh |
||||
|
So in Short -> it does not work. Details: To make it work with big files, I have changed: changed fopen to fopen64 changed fseek to fseeko64 changed ftell to ftello64 and also changed the int size to off64_t and all the place defined by using debug!! But it won't work!! Because it first reads the whole xml into buffer, and then reads node by node "xml tags" from the buffer. Reading 5 GB into that beffer hangs the computer, and is not a smart way! The below is the problamatic code: return (int)fread(buffer, 1, sizeToRead, File); Where sizeToRead is the size of the file. In our case 5 GB. Regards, Ali Awadh |
||||
|
|
){kind=link}