com.twitter.elephantbird.cascading2.scheme
Class CombinedSequenceFile

java.lang.Object
  cascading.scheme.Scheme<org.apache.hadoop.mapred.JobConf,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector,Object[],Void>
      cascading.scheme.hadoop.SequenceFile
          com.twitter.elephantbird.cascading2.scheme.CombinedSequenceFile

All Implemented Interfaces:

Serializable

public class CombinedSequenceFile
extends cascading.scheme.hadoop.SequenceFile

This scheme allows SequenceFile splits to be combined via the DelegateCombineFileInputFormat before it is read. It can be used to combine inputs for intermediate MR jobs in Cascading. To enable, set cascading.flowconnector.intermediateschemeclass to this class in the Hadoop configuration.

Author:

Akihiro Matsukawa

See Also:

Serialized Form

Constructor Summary

protected	CombinedSequenceFile()
	CombinedSequenceFile(cascading.tuple.Fields fields)

Method Summary

void	sourceConfInit(cascading.flow.FlowProcess<org.apache.hadoop.mapred.JobConf> flowProcess, cascading.tap.Tap<org.apache.hadoop.mapred.JobConf,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector> tap, org.apache.hadoop.mapred.JobConf conf)

Methods inherited from class cascading.scheme.hadoop.SequenceFile

sink, sinkConfInit, source, sourceCleanup, sourcePrepare

Methods inherited from class cascading.scheme.Scheme

equals, getNumSinkParts, getSinkFields, getSourceFields, getTrace, hashCode, isSink, isSource, isSymmetrical, presentSinkFields, presentSinkFieldsInternal, presentSourceFields, presentSourceFieldsInternal, retrieveSinkFields, retrieveSourceFields, setNumSinkParts, setSinkFields, setSourceFields, sinkCleanup, sinkPrepare, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

CombinedSequenceFile

protected CombinedSequenceFile()

CombinedSequenceFile

public CombinedSequenceFile(cascading.tuple.Fields fields)

Method Detail

sourceConfInit

public void sourceConfInit(cascading.flow.FlowProcess<org.apache.hadoop.mapred.JobConf> flowProcess,
                           cascading.tap.Tap<org.apache.hadoop.mapred.JobConf,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector> tap,
                           org.apache.hadoop.mapred.JobConf conf)

Overrides:

sourceConfInit in class cascading.scheme.hadoop.SequenceFile