Difference between revisions of "Windbag MapReduce Application"
| (3 intermediate revisions by the same user not shown) | |||
| Line 6: | Line 6: | ||
In this assignment we will try to determine who is the biggest windbag in a particular play with [https://en.wikipedia.org/wiki/Hamlet Hamlet] and [https://en.wikipedia.org/wiki/The_Importance_of_Being_Earnest The Importance of Being Earnest] serving as two pieces of data. | In this assignment we will try to determine who is the biggest windbag in a particular play with [https://en.wikipedia.org/wiki/Hamlet Hamlet] and [https://en.wikipedia.org/wiki/The_Importance_of_Being_Earnest The Importance of Being Earnest] serving as two pieces of data. | ||
| + | |||
| + | =Lecture= | ||
| + | {{CollapsibleYouTube|CSE 231s Lecture: Windbags|<youtube>orPUQqfOAJw</youtube>}} | ||
=Code To Implement= | =Code To Implement= | ||
==WindbagMapper== | ==WindbagMapper== | ||
| − | {{CodeToImplement|WindbagMapper|windbagLinePredicate<br/>map|mapreduce.apps.windbag. | + | {{CodeToImplement|WindbagMapper|windbagLinePredicate<br/>map|mapreduce.apps.windbag.exercise}} |
===constructor and instance variable=== | ===constructor and instance variable=== | ||
| Line 20: | Line 23: | ||
===map=== | ===map=== | ||
| − | Note: the WindBagMapper is constructed with an [https://docs.oracle.com/javase/8/docs/api/java/util/function/IntPredicate.html IntPredicate]. This IntPredicate's test method should be invoked to determine if a [https://www.cse.wustl.edu/~dennis.cosgrove/courses/cse231/ | + | Note: the WindBagMapper is constructed with an [https://docs.oracle.com/javase/8/docs/api/java/util/function/IntPredicate.html IntPredicate]. This IntPredicate's test method should be invoked to determine if a [https://www.cse.wustl.edu/~dennis.cosgrove/courses/cse231/current/apidocs/mapreduce/apps/play/core/PlayLine.html PlayLine]'s word count qualifies it as a windbag line. You can investigate the PlayLine class to see what information it contains. |
{{Sequential|public List<Map.Entry<PlayRole, Boolean>> map(PlayLine item)}} | {{Sequential|public List<Map.Entry<PlayRole, Boolean>> map(PlayLine item)}} | ||
| Line 36: | Line 39: | ||
{{Sequential|public Integer reduce(List<Boolean> container)}} | {{Sequential|public Integer reduce(List<Boolean> container)}} | ||
| − | {{Warning|This class inherits from ClassicReducer. Be sure to have completed that | + | {{Warning|This class inherits from ClassicReducer. Be sure to have completed that exercise first.}} |
==TruePortionClassicReducer== | ==TruePortionClassicReducer== | ||
| Line 50: | Line 53: | ||
{{Sequential|public Double reduce(List<Boolean> container)}} | {{Sequential|public Double reduce(List<Boolean> container)}} | ||
| − | {{Warning|This class inherits from ClassicReducer. Be sure to have completed that | + | {{Warning|This class inherits from ClassicReducer. Be sure to have completed that exercise first.}} |
| − | |||
=Client= | =Client= | ||
{{Client|WindbagApp|mapreduce.apps.windbag.client}} | {{Client|WindbagApp|mapreduce.apps.windbag.client}} | ||
| − | {{CollapsibleConsole|HAMLET|<pre> | + | {{CollapsibleConsole|HAMLET|<pre style="border: 0px; background: #000; color:#fff;"> |
COUNT | COUNT | ||
=========================== | =========================== | ||
| Line 93: | Line 95: | ||
QUEEN: 0.014493</pre>}} | QUEEN: 0.014493</pre>}} | ||
| − | {{CollapsibleConsole|THE_IMPORTANCE_OF_BEING_EARNEST|<pre> COUNT | + | {{CollapsibleConsole|THE_IMPORTANCE_OF_BEING_EARNEST|<pre style="border: 0px; background: #000; color:#fff;"> COUNT |
=========================== | =========================== | ||
LADY BRACKNELL: 7 | LADY BRACKNELL: 7 | ||
| Line 110: | Line 112: | ||
JACK: 0.013825 | JACK: 0.013825 | ||
ALGERNON: 0.009950</pre>}} | ALGERNON: 0.009950</pre>}} | ||
| − | |||
=Testing Your Solution= | =Testing Your Solution= | ||
Latest revision as of 09:20, 7 March 2024
Contents
Motivation
A simple Mapper and two simple ClassicReducers reinforce our understanding of how pieces of the MapReduce puzzle can be reused. Further, we will reduce the value type of List<Boolean> to both an Integer and a Double in separate Reducers.
Background
A windbag is an exhaustively talkative person. Act I Scene 3 of Hamlet has a famous example.
In this assignment we will try to determine who is the biggest windbag in a particular play with Hamlet and The Importance of Being Earnest serving as two pieces of data.
Lecture
| Video: CSE 231s Lecture: Windbags |
|---|
Code To Implement
WindbagMapper
| class: | WindbagMapper.java |  |
| methods: | windbagLinePredicate map |
|
| package: | mapreduce.apps.windbag.exercise | |
| source folder: | student/src/main/java |
constructor and instance variable
method: public WindbagMapper(IntPredicate windbagLinePredicate) 
(sequential implementation only)
Hang onto the value of the windbagLinePredicate parameter as an instance variable for later use.
windbagLinePredicate
This method exists largely as a reminder to use the windbagLinePredicate passed to the constructor.
map
Note: the WindBagMapper is constructed with an IntPredicate. This IntPredicate's test method should be invoked to determine if a PlayLine's word count qualifies it as a windbag line. You can investigate the PlayLine class to see what information it contains.
method: public List<Map.Entry<PlayRole, Boolean>> map(PlayLine item) (sequential implementation only)
TrueCountClassicReducer
This Collector reduces to the number of emitted trues.
For example, if a total of eight (8) values were emitted for a particular key and three (3) of them were true:
[false, true, false, false, true, true, false, false]
then the reduced value would be 3
| class: | TrueCountClassicReducer.java | |
| methods: | reduce | |
| package: | mapreduce.apps.windbag.exercise | |
| source folder: | student/src/main/java |
method: public Integer reduce(List<Boolean> container) (sequential implementation only)
 Warning:This class inherits from ClassicReducer. Be sure to have completed that exercise first. |
TruePortionClassicReducer
This Collector reduces to the portion of emitted trues.
For example, if a total of eight (8) values were emitted for a particular key and three (3) of them were true:
[false, true, false, false, true, true, false, false]
then the reduced value would be 0.375
| class: | TruePortionClassicReducer.java | |
| methods: | reduce | |
| package: | mapreduce.apps.windbag.exercise | |
| source folder: | student/src/main/java |
method: public Double reduce(List<Boolean> container) (sequential implementation only)
| Warning:This class inherits from ClassicReducer. Be sure to have completed that exercise first. |
Client
| class: | WindbagApp.java | CLIENT |
| package: | mapreduce.apps.windbag.client | |
| source folder: | student/src//java |
| HAMLET |
|---|
COUNT
===========================
HAMLET: 27
KING: 14
POLONIUS: 7
HORATIO: 5
OPHELIA: 2
GHOST: 2
FIRST PLAYER: 2
LAERTES: 1
ROSENCRANTZ: 1
VOLTEMAND: 1
PLAYER KING: 1
GENTLEMAN: 1
PLAYER QUEEN: 1
QUEEN: 1
PORTION
===========================
VOLTEMAND: 1.000000
GENTLEMAN: 0.333333
PLAYER KING: 0.250000
FIRST PLAYER: 0.250000
PLAYER QUEEN: 0.200000
GHOST: 0.142857
KING: 0.137255
POLONIUS: 0.081395
HAMLET: 0.075419
HORATIO: 0.046729
OPHELIA: 0.034483
ROSENCRANTZ: 0.022222
LAERTES: 0.016129
QUEEN: 0.014493
|
| THE_IMPORTANCE_OF_BEING_EARNEST |
|---|
COUNT
===========================
LADY BRACKNELL: 7
JACK: 3
ALGERNON: 2
GWENDOLEN: 2
MISS PRISM: 1
CHASUBLE: 1
PORTION
===========================
LADY BRACKNELL: 0.083333
MISS PRISM: 0.024390
CHASUBLE: 0.023810
GWENDOLEN: 0.019802
JACK: 0.013825
ALGERNON: 0.009950
|
Testing Your Solution
| class: | _WindbagTestSuite.java |  |
| package: | mapreduce.apps.windbag.exercise | |
| source folder: | testing/src/test/java |
Pledge, Acknowledgments, Citations
| file: | mapreduce-windbag-app-pledge-acknowledgments-citations.txt |
More info about the Honor Pledge
