Как найти строку в файле java

Fast comes at a price…. code complexity and perhaps readability.

Assuming that your code produces the right results now…. and it is a big assumption because:

• it expects the word to be at the beginning/end of the line, or surrounded by spaces (not commas, punctuation, etc.)
• it does not look for the word inside another string, it will match ‘are’, but not ‘bare’.

OK, a much faster way (keeping it as Java), is to do the following:

1. Convert the search string (‘are’) to a byte-array in the same encoding as the file.
2. Open a memory-mapped byte-buffer from a File-Channel on the file.
3. Scan the ByteBuffer, looking for matches to the search byte-array
4. count the newlines as you go.
5. close the ByteBuffer

If the file is larger than your memory, you will have to re-position the byte-buffer occasionally. I recommend using a emopry-mapped size of about 4MB plus the size of the search-string. That way you can search the 4MB window, and then start the next window at the next 4mb boundary.

Once you get in to it, it will make sense.

This system will be fast because you will never have to copy the file’s data in to Java. Everything will actually happen in the native side of things.

There is a lot to read to make it work.

I would start with a tutorial….

• StudyTrails
• Yaldix
• LinuxTopia

of course, if you want really fast, use grep.

Here is some example code that may get you started:

import java.io.IOException;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.channels.FileChannel.MapMode;
import java.nio.charset.StandardCharsets;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;


public class NIOGrep {

    public static void main(String[] args) throws IOException {
        if (args.length != 2) {
            throw new IllegalArgumentException();
        }
        String grepfor = args[0];
        Path path = Paths.get(args[1]);

        String report = searchFor(grepfor, path);

        System.out.println(report);

    }

    private static final int MAPSIZE = 4 * 1024 ; // 4K - make this * 1024 to 4MB in a real system.

    private static String searchFor(String grepfor, Path path) throws IOException {
        final byte[] tosearch = grepfor.getBytes(StandardCharsets.UTF_8);
        StringBuilder report = new StringBuilder();
        int padding = 1; // need to scan 1 character ahead in case it is a word boundary.
        int linecount = 0;
        int matches = 0;
        boolean inword = false;
        boolean scantolineend = false;
        try (FileChannel channel = FileChannel.open(path, StandardOpenOption.READ)) {
            final long length = channel.size();
            int pos = 0;
            while (pos < length) {
                long remaining = length - pos;
                // int conversion is safe because of a safe MAPSIZE.. Assume a reaosnably sized tosearch.
                int trymap = MAPSIZE + tosearch.length + padding;
                int tomap = (int)Math.min(trymap, remaining);
                // different limits depending on whether we are the last mapped segment.
                int limit = trymap == tomap ? MAPSIZE : (tomap - tosearch.length);
                MappedByteBuffer buffer = channel.map(MapMode.READ_ONLY, pos, tomap);
                System.out.println("Mapped from " + pos + " for " + tomap);
                pos += (trymap == tomap) ? MAPSIZE : tomap;
                for (int i = 0; i < limit; i++) {
                    final byte b = buffer.get(i);
                    if (scantolineend) {
                        if (b == 'n') {
                            scantolineend = false;
                            inword = false;
                            linecount ++;
                        }
                    } else if (b == 'n') {
                        linecount++;
                        inword = false;
                    } else if (b == 'r' || b == ' ') {
                        inword = false;
                    } else if (!inword) {
                        if (wordMatch(buffer, i, tomap, tosearch)) {
                            matches++;
                            i += tosearch.length - 1;
                            if (report.length() > 0) {
                                report.append(", ");
                            }
                            report.append(linecount);
                            scantolineend = true;
                        } else {
                            inword = true;
                        }
                    }
                }
            }
        }
        return "Times found at--" + matches + "nWord found at--" + report;
    }

    private static boolean wordMatch(MappedByteBuffer buffer, int pos, int tomap, byte[] tosearch) {
        //assume at valid word start.
        for (int i = 0; i < tosearch.length; i++) {
            if (tosearch[i] != buffer.get(pos + i)) {
                return false;
            }
        }
        byte nxt = (pos + tosearch.length) == tomap ? (byte)' ' : buffer.get(pos + tosearch.length); 
        return nxt == ' ' || nxt == 'n' || nxt == 'r';
    }
}

Untried, but probably the fastest mechanism is to first, take your search key and encode it like the file.

For example, if you know the file is UTF-8, take your key and encode it from a String (which it UTF-16) in to a byte array that is UTF-8. This is important because by encoding down to the file representation, you’re only encoding the key. Using standard Java Readers goes the other way — converts the file to UTF-16.

Now that you have a proper key, in bytes, use NIO to create a MappedByteBuffer for the file. This maps the file in to the virtual memory space.

Finally, implement a Boyer-Moore algorithm for string search, using the bytes of the key against the bytes of the file via the mapped region,

There may well be a faster way, but this solves a bulk of the problems with searching a text file in Java. It leverages the VM to avoid copying large chunks of the file, and it skips the conversion step of whatever encoding the file is in to UTF-16, which Java uses internally.