protected static class ThrottledFetcher.ThrottledConnection extends Object implements IThrottledConnection
Modifier and Type | Field and Description |
---|---|
protected ThrottledFetcher.AbortChecker |
abortChecker
Abort checker
|
protected org.apache.http.conn.HttpClientConnectionManager |
connectionManager
The client connection manager
|
protected IConnectionThrottler |
connectionThrottler
The throttling object we use to track connections
|
protected int |
connectionTimeoutMilliseconds
Connection timeout in milliseconds
|
protected org.apache.http.client.methods.HttpRequestBase |
executeMethod
The method object
|
protected long |
fetchCounter
The current bytes in the current fetch
|
protected IFetchThrottler |
fetchThrottler
The throttling object we use to track fetches
|
protected String |
fetchType
The kind of fetch we are doing
|
protected org.apache.http.client.HttpClient |
httpClient
The httpclient
|
protected ThrottledFetcher.ExecuteMethodThread |
methodThread
The thread that is actually doing the work
|
protected String |
myUrl
The current URL being fetched
|
protected String |
serverName
The server fqdn
|
protected long |
startFetchTime
The start-fetch time
|
protected int |
statusCode
The status code fetched, if any
|
protected boolean |
threadStarted
Set if thread has been started
|
protected Throwable |
throwable
The error trace, if any
|
_rcsid, FETCH_BAD_URI, FETCH_CIRCULAR_REDIRECT, FETCH_IO_ERROR, FETCH_NOT_TRIED, FETCH_SEQUENCE_ERROR, FETCH_UNKNOWN_ERROR, STATUS_NOCHANGE, STATUS_OK, STATUS_PAGEERROR, STATUS_SITEERROR
Constructor and Description |
---|
ThrottledFetcher.ThrottledConnection(String serverName,
IConnectionThrottler connectionThrottler,
int connectionTimeoutMilliseconds,
int connectionLimit,
String proxyHost,
int proxyPort,
String proxyAuthDomain,
String proxyAuthUsername,
String proxyAuthPassword,
IAbortActivity activities)
Constructor.
|
Modifier and Type | Method and Description |
---|---|
void |
beginFetch(String fetchType)
Begin the fetch process.
|
void |
close()
Close the connection.
|
void |
doneFetch(IProcessActivity activities)
Done with the fetch.
|
int |
executeFetch(String protocol,
int port,
String urlPath,
String userAgent,
String from,
String lastETag,
String lastModified)
Execute the fetch and get the return code.
|
InputStream |
getResponseBodyStream()
Get the response input stream.
|
int |
getResponseCode()
Get the http response code.
|
String |
getResponseHeader(String headerName)
Get a specified response header, if it exists.
|
void |
logFetchCount(int count)
Log the fetch of a number of bytes.
|
protected final String serverName
protected final IConnectionThrottler connectionThrottler
protected final IFetchThrottler fetchThrottler
protected final int connectionTimeoutMilliseconds
protected final org.apache.http.conn.HttpClientConnectionManager connectionManager
protected final org.apache.http.client.HttpClient httpClient
protected org.apache.http.client.methods.HttpRequestBase executeMethod
protected long startFetchTime
protected Throwable throwable
protected String myUrl
protected int statusCode
protected String fetchType
protected long fetchCounter
protected ThrottledFetcher.ExecuteMethodThread methodThread
protected boolean threadStarted
protected final ThrottledFetcher.AbortChecker abortChecker
public ThrottledFetcher.ThrottledConnection(String serverName, IConnectionThrottler connectionThrottler, int connectionTimeoutMilliseconds, int connectionLimit, String proxyHost, int proxyPort, String proxyAuthDomain, String proxyAuthUsername, String proxyAuthPassword, IAbortActivity activities) throws ManifoldCFException, ServiceInterruption
public void beginFetch(String fetchType) throws ManifoldCFException, ServiceInterruption
beginFetch
in interface IThrottledConnection
fetchType
- is a short descriptive string describing the kind of fetch being requested. This
is used solely for logging purposes.ManifoldCFException
ServiceInterruption
public void logFetchCount(int count)
public int executeFetch(String protocol, int port, String urlPath, String userAgent, String from, String lastETag, String lastModified) throws ManifoldCFException, ServiceInterruption
executeFetch
in interface IThrottledConnection
protocol
- is the protocol to use to perform the access, e.g. "http"port
- is the port to use to perform the access, where -1 means "use the default"urlPath
- is the path part of the url, e.g. "/robots.txt"userAgent
- is the value of the userAgent header to use.from
- is the value of the from header to use.proxyHost
- is the proxy host, or null if none.proxyPort
- is the proxy port, or -1 if none.proxyAuthDomain
- is the proxy authentication domain, or null.proxyAuthUsername
- is the proxy authentication user name, or null.proxyAuthPassword
- is the proxy authentication password, or null.lastETag
- is the requested lastETag header value.lastModified
- is the requested lastModified header value.ManifoldCFException
ServiceInterruption
public int getResponseCode() throws ManifoldCFException, ServiceInterruption
getResponseCode
in interface IThrottledConnection
ManifoldCFException
ServiceInterruption
public InputStream getResponseBodyStream() throws ManifoldCFException, ServiceInterruption
getResponseBodyStream
in interface IThrottledConnection
ManifoldCFException
ServiceInterruption
public String getResponseHeader(String headerName) throws ManifoldCFException, ServiceInterruption
getResponseHeader
in interface IThrottledConnection
headerName
- is the name of the header.ManifoldCFException
ServiceInterruption
public void doneFetch(IProcessActivity activities) throws ManifoldCFException
doneFetch
in interface IThrottledConnection
ManifoldCFException
public void close() throws ManifoldCFException
close
in interface IThrottledConnection
ManifoldCFException