Multithread and file buffering problems
I've written a dual thread program that does the following:
Thread1 --> Read From Serial port and save to buffer
Thread2 --> Read from buffer and save to file
For the 2nd thread, i'm using CFile class functions to handle file I/O. I'm experiencing synchronisation issues here.
I do not know how to use fstream class functions - will it solve my synchronisation problems?
Thanks!
[440 byte] By [
tkgsean] at [2007-11-18 1:38:19]
# 2 Re: Multithread and file buffering problems
Here's the summarised code :
//------Thread1-------
...
packetsCompleted=0;
for(int i=0;i<=packetSize;i++)
{
...
MySerialCom.ReadByte(szBuffer[i])
if(i==packetSize && packetsCompleted<totalPackets)
{
packetsCompleted++;
i=0;
}
}
...
//------Thread2------
while(packetsCompleted != totalPackets)
{
filename = "testResult.txt";
myFile.open(filename...);
myFile.seek(packetsCompleted*packetSize)
myFile.write(szBuffer,packetSize)
}
...
I've been testing it by reading a text file sent from the serial port,
Contents of test.txt:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
ccccccccccccccccccccccccccccccccccccccccccccc
dddddddddddddddddddddddddddddddddd
eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee
...
zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
Now I set the buffersize of szBuffer to be half the size of the "test.txt" file, so that szBuffer will have to cycle itself once.
Now the "testResult.txt" will look something like:
a aaaaaaaaaaaaaaaaaaaaaaaaaaaa
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
ccccccccccccccccccccccccccccccccccccccccccccc
dddddddddddddddddddddddddddddddddd
eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee
...
zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
*Realise the missing aplhabets in the first line!
I've done all these because the final program need to read a large file from the serial port without pause, and the szbuffer will only be abt 100KB.
Question
----
1. Is this a synchronisation problem?
2. Will using fstream help?
3. Other alternatives?
Thanks Plenty...
# 3 Re: Multithread and file buffering problems
I would use std::vector's built in push and pop to create a FIFO buffer.
std::vector<BYTE[256]> data;
-- Thread 1
BYTE buffer[256];
while( something )
{
for( int i = 0; i < 256;i++ )
{
MySerialCom.ReadByte(buffer[i]);
}
data.push_back( buffer );
}
-- Thread 2
/// Open file.
while( something )
{
if( !m_data.empty() )
{
BYTE buffer[256];
data.pop_front( buffer );
/// write buffer
}
sleep( 100 );
}
# 4 Re: Multithread and file buffering problems
Originally posted by mwilliamson
I would use std::vector's built in push and pop to create a FIFO buffer.
That would not help much. Even the STL does not necessarely provide synchronization automatically (I am not aware of any STL implementation which does it).
Therefore the root problem is not solved by using different methods ('CFile', 'std::vector' etc.) but rather by providing some kind of synchronization (mutex).
# 6 Re: Multithread and file buffering problems
Originally posted by mwilliamson
Why not, did you look at my code. I'm fairly certain it would work with out problems.
Well I did look at your code...but I does not change my previous statements...
There is still the same problem as before...and it is called synchronization...
Based on your example consider the following:
Thread2 is doing a 'pop_front()' on the global buffer. After getting 100 bytes a context switch will occur and Thread1 will get execution time. Thread1 will call 'push_back()'. This call will cause a re-allocation of the buffer since the old one is too small. It does it and after that a context switch will occur giving thread2 CPU time. Remember the thread was left within the 'pop_front()' call after the first 100 bytes. Right now the thread will just continue to read the 101th byte. What will happen...basically you will see a pop-up window telling you that an exception 0x00000005 has occured...
The reason in this case would be that Thread2 still expects the data at the same memory location as before. Since the call to 'push_back()' moved the content to a different memory spot due to reallocation the old memory became invalid. If 'pop_front()' and 'push_back()' would be atomic functions (which they are not - at least that would be new to me) then there would not be a problem with this sample code (although it is still dangerous in general). I agree though that this can be handled by the implementation of 'pop_front()' and 'push_back()' but you never know how these functions are implemented internally...
As I have pointed out...as soon as you have multiple threads sharing a resource you need to take care about the correct synchronization to it. This is valid even if you use STL since the standard does not guarantee multithread safety for its containers. It is the job of the implementers of the different STL versions to do it or not. And as I said I did not have seen one which basically does it yet...
