com.etsy.cascading.tap.local

Class LocalTap<SourceCtx,SinkCtx>

java.lang.Object

cascading.tap.Tap<java.util.Properties,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector>

com.etsy.cascading.tap.local.LocalTap<SourceCtx,SinkCtx>

All Implemented Interfaces:

cascading.flow.FlowElement, cascading.util.Traceable, java.io.Serializable

public class LocalTap<SourceCtx,SinkCtx>
extends cascading.tap.Tap<java.util.Properties,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector>

A shunt around the restrictions on taps that can run under the LocalFlowProcess. This class delegates to Lfs for all work, yet can run in Cascading local mode. Due to what appears to be a bug in LocalFlowStep#initTaps, schemes are not allowed to side-effect the properties passed to sourceConfInit and sinkConfInit. This appears to be a non-issue because local schemes never set any properties. overwriteProperties can likely be removed, but is kept in case custom local schemes eventually set properties (in a future release of Cascading where that's possible).

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
LocalTap(java.lang.String path, cascading.scheme.Scheme<org.apache.hadoop.mapred.JobConf,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector,SourceCtx,SinkCtx> scheme)
LocalTap(java.lang.String path, cascading.scheme.Scheme<org.apache.hadoop.mapred.JobConf,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector,SourceCtx,SinkCtx> scheme, cascading.tap.SinkMode sinkMode)

Method Summary

Modifier and Type	Method and Description
boolean	createResource(java.util.Properties conf)
boolean	deleteResource(java.util.Properties conf)
boolean	equals(java.lang.Object object)
java.lang.String	getIdentifier()
long	getModifiedTime(java.util.Properties conf)
int	hashCode()
cascading.tuple.TupleEntryIterator	openForRead(cascading.flow.FlowProcess<java.util.Properties> flowProcess, org.apache.hadoop.mapred.RecordReader input)
cascading.tuple.TupleEntryCollector	openForWrite(cascading.flow.FlowProcess<java.util.Properties> flowProcess, org.apache.hadoop.mapred.OutputCollector output)
boolean	resourceExists(java.util.Properties conf)
java.lang.String	toString()

Methods inherited from class cascading.tap.Tap

commitResource, createResource, deleteResource, flowConfInit, getConfigDef, getFullIdentifier, getFullIdentifier, getModifiedTime, getScheme, getSinkFields, getSinkMode, getSourceFields, getStepConfigDef, getTrace, hasConfigDef, hasStepConfigDef, id, isEquivalentTo, isKeep, isReplace, isSink, isSource, isTemporary, isUpdate, openForRead, openForWrite, outgoingScopeFor, presentSinkFields, presentSourceFields, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, resourceExists, retrieveSinkFields, retrieveSourceFields, rollbackResource, setScheme, sinkConfInit, sourceConfInit, taps

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

LocalTap

public LocalTap(java.lang.String path,
                cascading.scheme.Scheme<org.apache.hadoop.mapred.JobConf,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector,SourceCtx,SinkCtx> scheme,
                cascading.tap.SinkMode sinkMode)

LocalTap

public LocalTap(java.lang.String path,
                cascading.scheme.Scheme<org.apache.hadoop.mapred.JobConf,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector,SourceCtx,SinkCtx> scheme)

Method Detail

getIdentifier

public java.lang.String getIdentifier()

Specified by:

getIdentifier in class cascading.tap.Tap<java.util.Properties,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector>

openForRead

public cascading.tuple.TupleEntryIterator openForRead(cascading.flow.FlowProcess<java.util.Properties> flowProcess,
                                                      org.apache.hadoop.mapred.RecordReader input)
                                               throws java.io.IOException

Specified by:

openForRead in class cascading.tap.Tap<java.util.Properties,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector>

Throws:

java.io.IOException

openForWrite

public cascading.tuple.TupleEntryCollector openForWrite(cascading.flow.FlowProcess<java.util.Properties> flowProcess,
                                                        org.apache.hadoop.mapred.OutputCollector output)
                                                 throws java.io.IOException

Specified by:

openForWrite in class cascading.tap.Tap<java.util.Properties,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector>

Throws:

java.io.IOException

createResource

public boolean createResource(java.util.Properties conf)
                       throws java.io.IOException

Specified by:

createResource in class cascading.tap.Tap<java.util.Properties,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector>

Throws:

java.io.IOException

deleteResource

public boolean deleteResource(java.util.Properties conf)
                       throws java.io.IOException

Specified by:

deleteResource in class cascading.tap.Tap<java.util.Properties,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector>

Throws:

java.io.IOException

resourceExists

public boolean resourceExists(java.util.Properties conf)
                       throws java.io.IOException

Specified by:

resourceExists in class cascading.tap.Tap<java.util.Properties,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector>

Throws:

java.io.IOException

getModifiedTime

public long getModifiedTime(java.util.Properties conf)
                     throws java.io.IOException

Specified by:

getModifiedTime in class cascading.tap.Tap<java.util.Properties,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector>

Throws:

java.io.IOException

equals

public boolean equals(java.lang.Object object)

Overrides:

equals in class cascading.tap.Tap<java.util.Properties,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector>

hashCode

public int hashCode()

Overrides:

hashCode in class cascading.tap.Tap<java.util.Properties,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector>

toString

public java.lang.String toString()

Overrides:

toString in class cascading.tap.Tap<java.util.Properties,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector>

See Also:

Object.toString()