Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Efficient way of Reading a very large text file from a specific line number to the end in C#

I have a requirement where I need to read a very large text file (~2 Gb), but only starting from a specific line number till the end of the file.

I can not load the whole text in memory due to performance issues. So I have used StreamReader. But I noticed that there is no easy way to start the "reading" from a specific line number, Rather what I have done is I have started to read the file from line 1, and ignoring all the lines before I reach my desired line number.

Is this the correct approach ? This is what I have tried. Is there a better way to achieving this?

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

static string ReadLogFileFromSpecificLine(int LineNumber)
    {
        string content = null;

        using (StreamReader sr = new StreamReader(LogFilePath))
        {
            sr.ReadLine();
            int currentLineNumber = 0;

            string line;
            while ((line = sr.ReadLine()) != null)
            {
                currentLineNumber++;
                if(currentLineNumber >= LineNumber - 1)
                {
                    content += line + "\n";
                }
            }                
        }
        return content;
    }

>Solution :

This is the correct approach. There is no algorithm to figure out the offset of a particular line and seek to it.

You may be able to squeeze a little more performance out of it by have two loops. Once the starting line is reached, you could move to the second loop, which wouldn’t need to check the line number. But that would impact performance only minimally at best.

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading