public class ThrottledFetcher extends Object
Modifier and Type | Class and Description |
---|---|
protected static class |
ThrottledFetcher.AbortChecker
This class furnishes an abort signal whenever the job activity says it should.
|
protected static class |
ThrottledFetcher.ExecuteMethodThread
This thread does the actual socket communication with the server.
|
protected static class |
ThrottledFetcher.ThrottledConnection
This class represents an established connection to a URL.
|
protected static class |
ThrottledFetcher.ThrottledInputstream
This class throttles an input stream based on the specified byte rate parameters.
|
Modifier and Type | Field and Description |
---|---|
static String |
_rcsid |
protected static int |
globalHandleCount
This counter keeps track of the total outstanding handles across everything, because we do try to control that
|
protected static Integer |
globalHandleCounterLock
This is the lock object for that global handle counter
|
protected static int |
READ_CHUNK_LENGTH
The read chunk length
|
protected static boolean |
recordEverything
This flag determines whether we record everything to the disk, as a means of doing a web snapshot
|
protected int |
refCount
Reference count for how many connections to this pool there are
|
protected Map<String,IConnectionThrottler> |
serverMap
This hash maps the server string (without port) to a pool throttling object, where
we can track the statistics and make sure we throttle appropriately
|
Constructor and Description |
---|
ThrottledFetcher()
Constructor.
|
Modifier and Type | Method and Description |
---|---|
IThrottledConnection |
createConnection(IThreadContext threadContext,
String throttleGroupName,
String serverName,
int connectionLimit,
int connectionTimeoutMilliseconds,
String proxyHost,
int proxyPort,
String proxyAuthDomain,
String proxyAuthUsername,
String proxyAuthPassword,
IAbortActivity activities)
Establish a connection to a specified URL.
|
void |
noteConnectionEstablished()
Note that there is a repository connection that is using this object.
|
void |
noteConnectionReleased()
Connection pool no longer needed.
|
void |
poll()
Poll.
|
protected static void |
registerGlobalHandle(int maxHandles)
Note that we're about to need a handle (and make sure we have enough)
|
protected static void |
releaseGlobalHandle()
Note that we're done with a handle (so we can free it)
|
public static final String _rcsid
protected static final boolean recordEverything
protected static final int READ_CHUNK_LENGTH
protected static int globalHandleCount
protected static Integer globalHandleCounterLock
protected final Map<String,IConnectionThrottler> serverMap
protected int refCount
protected static void registerGlobalHandle(int maxHandles) throws ManifoldCFException
ManifoldCFException
protected static void releaseGlobalHandle()
public IThrottledConnection createConnection(IThreadContext threadContext, String throttleGroupName, String serverName, int connectionLimit, int connectionTimeoutMilliseconds, String proxyHost, int proxyPort, String proxyAuthDomain, String proxyAuthUsername, String proxyAuthPassword, IAbortActivity activities) throws ManifoldCFException, ServiceInterruption
serverName
- is the FQDN of the server, e.g. foo.metacarta.comconnectionLimit
- is the maximum desired outstanding connections at any one time.connectionTimeoutMilliseconds
- is the number of milliseconds to wait for the connection before timing out.ManifoldCFException
ServiceInterruption
public void poll() throws ManifoldCFException
ManifoldCFException
public void noteConnectionEstablished()
public void noteConnectionReleased()