Monday, April 16, 2012

Java multithreading and files

I'm working on project. One part of it is read given folders files.
Program travels into deep and collects filenames and other info which i wrap into my own DFile class, and puts it into collection for further work.
It worked when was singlethreaded (using recursive read), but I want to do that in multithreading perspective, ignoring the thing that disk IO and multithreading won't increase performance. I want it for learning purpose.



So far, I've been jumping from one decision to another, changing plans how it will be and can't get it good. Your help would be appreciated.



What I want, that I supply root folder name, and my program runs several minithreads (user defined number of threads for this purpose), each thread reads given folders content:
- When it finds file, wraps it into DFile and puts into shared between threads collection
- When it finds folder, puts folder (as File object) into jobQueue, for other available thread to take work on it.



I can't get this system correctly. I've been changing code, puting idea what classes should be from one class with static collections to many.
So far few classes I am listing here:



DirectoryCrawler http://pastebin.com/8tVGpGT9



Won't publish rest of my work (maybe in other topic, because purpose of the program absolutely not covered here). Program should read folder and make a list of files in it, then sort it (where I'll probably use multithreading too), then search for same hashed files and there's constantly working thread for writing those equal file groups into result file. I don't need to gain any performance, files gonna be small, as at first I was working on speed, I don't need it now.



Any help regarding design of reading would be appreciated



EDIT:



So much of headache :((. Doesn't work correctly :( Here so far:
crawler (like a minithread for reading one folder, found files goes to fileList which is in other class, and folders to queue) pastebin. com/AkJLAUhD



scanner class (Don't even know should it be runnable or no). DirectoryScanner (main, should control crawlers, hold main filelist) pastebin. com/2abGMgG9 .



DFile itself pastebin. com/8uqPWh6Z (something became wrong with hashing, now when sorting all get same hash.. worked .. (hashing is for other task unrelated)) .



Filelist http:// pastebin. com/Q2yM6ZwS



testcode:



DirectoryScanner reader = new DirectoryScanner(4);
for (int i = 0; i < 4; i ++) {
reader.runTask(new DirectoryCrawler("myroot", reader));
}
try {
reader.kill();
while (!reader.isDone()) {
System.out.println("notdone");
}
reader.getFileList().print();
}


myroot is a folder with some files for test



Anything, i can't even think of should scanner be itself runnable, or only crawlers. Because while scanning I actualy don't want to start doing other stuff like sorting (because nothing to sort while not gathered all files) ..





No comments:

Post a Comment