Difference between revisions of "MapReduce Reducer Assignment"
Jump to navigation
Jump to search
Line 3: | Line 3: | ||
=Background= | =Background= | ||
− | [https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collector.html interface Collector<T,A,R>] | + | After much debate, we decided to go with [https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collector.html interface Collector<T,A,R>] from the standard Java streams framework as the basis for the MapReduce assignment. |
+ | |||
+ | It might have been slightly less confusing at the outset if we provided our own custom interface. Imaging the mythical non-existant interface below: | ||
+ | |||
+ | <nowiki>public interface Reducer<V, A, R> { | ||
+ | A createMutableContainer(); | ||
+ | void accumulate(A container, V item); | ||
+ | A combine(A containerA, A containerB); | ||
+ | R reduce(A container); | ||
+ | }</nowiki> | ||
+ | |||
+ | |||
[https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/mutable/MutableInt.html class MutableInt] | [https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/mutable/MutableInt.html class MutableInt] |
Revision as of 17:50, 22 February 2018
Contents
Motivation
Collector is fundamental to the MapReduce Framework lab.
Background
After much debate, we decided to go with interface Collector<T,A,R> from the standard Java streams framework as the basis for the MapReduce assignment.
It might have been slightly less confusing at the outset if we provided our own custom interface. Imaging the mythical non-existant interface below:
public interface Reducer<V, A, R> { A createMutableContainer(); void accumulate(A container, V item); A combine(A containerA, A containerB); R reduce(A container); }
Mythical Reducer:
public class CollectorReducerAdapter<V, A, R> implements Reducer<V, A, R> { private final Collector<V,A,R> collector; public CollectorReducerAdapter(Collector<V,A,R> collector) { this.collector = collector; } @Override public A createMutableContainer() { return collector.supplier().get(); } @Override public void accumulate(A container, V item) { collector.accumulator().accept(container, item); } @Override public A combine(A containerA, A containerB) { return collector.combiner().apply(containerA, containerB); } @Override public R reduce(A container) { return collector.finisher().apply(container); } }
Code To Implement
ClassicReducer
class: | ClassicReducer.java | |
methods: | supplier accumulator combiner |
|
package: | mapreduce.collector.studio | |
source folder: | student/src/main/java |
IntSumCollector
class: | IntSumCollector.java | |
methods: | supplier accumulator combiner finisher |
|
package: | mapreduce.collector.intsum.studio | |
source folder: | student/src/main/java |
Testing Your Solution
Correctness
class: | CollectorStudioTestSuite.java | |
package: | mapreduce | |
source folder: | testing/src/test/java |