bass.crawler
Class BandDistribution

java.lang.Object
  bass.crawler.BandDistribution

public class BandDistribution
extends java.lang.Object

Used for distributing band IDs to several server threads based on a given integer array of IDs. Also keeps track of which bands weren't crawled (because of errors etc.) in an hashtable to provide those IDs after the initial band IDs were distributed.

Author:

Fabian Winkhardt, Thomas Klein

Field Summary

private int[]	bandArray
private int	bandIndex
private java.util.Hashtable<java.lang.Integer,java.lang.Boolean>	errorHash
private int	errorIndex
private long	startTime
private long	waitTime

Constructor Summary

BandDistribution(int[] bandArray)

Method Summary

int	getBandCount()
int	getID() Returns a band ID to crawl
boolean	getStatus(int id)
void	setStatus(int id, boolean pageStatus) Using a hashtable to save the pages we already crawled and the ones with errors to check them later again
void	writeMissingBands(java.lang.String mode) Writes the missing band IDs into a text file once every ID (error-hash as well) has been distributed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

bandIndex

private int bandIndex

errorIndex

private int errorIndex

bandArray

private int[] bandArray

errorHash

private java.util.Hashtable<java.lang.Integer,java.lang.Boolean> errorHash

startTime

private long startTime

waitTime

private long waitTime

Constructor Detail

BandDistribution

public BandDistribution(int[] bandArray)

Method Detail

getID

public int getID()

Returns a band ID to crawl

Returns:

int - a band ID, which has to be crawled

setStatus

public void setStatus(int id,
                      boolean pageStatus)

Using a hashtable to save the pages we already crawled and the ones with errors to check them later again

Parameters:

pageNumber -

pageStatus -

getStatus

public boolean getStatus(int id)

getBandCount

public int getBandCount()

Returns:

the number of bands in this object

writeMissingBands

public void writeMissingBands(java.lang.String mode)

Writes the missing band IDs into a text file once every ID (error-hash as well) has been distributed.

Parameters:

mode - - bands, profiles or friends