Difference between revisions of "Windbag MapReduce Application"

From CSE231 Wiki
Jump to navigation Jump to search
 
(7 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
=Motivation=
 
=Motivation=
−
A simple Mapper and two simple Collectors reinforce our understanding of how pieces of the MapReduce puzzle can be reused.  Further, we will reduce the value type of Boolean to both an Integer and a Double in two different Collectors.
+
A simple Mapper and two simple ClassicReducers reinforce our understanding of how pieces of the MapReduce puzzle can be reused.  Further, we will reduce the value type of List<Boolean> to both an Integer and a Double in separate Reducers.
  
 
=Background=
 
=Background=
−
A windbag is [https://www.merriam-webster.com/dictionary/windbag an exhaustively talkative person].  In this studio we will try to find who is the biggest windbag in [https://en.wikipedia.org/wiki/Hamlet Hamlet].
+
A windbag is [https://www.merriam-webster.com/dictionary/windbag an exhaustively talkative person].  [http://shakespeare.mit.edu/hamlet/hamlet.1.3.html Act I Scene 3 of Hamlet] has a [https://en.wikipedia.org/wiki/Polonius#Famous_lines famous example].
  
−
[http://shakespeare-online.com/plays/hamlet_1_3.html Polonius' Advice To Laertes]
+
In this assignment we will try to determine who is the biggest windbag in a particular play with [https://en.wikipedia.org/wiki/Hamlet Hamlet] and [https://en.wikipedia.org/wiki/The_Importance_of_Being_Earnest The Importance of Being Earnest] serving as two pieces of data.
 +
 
 +
=Lecture=
 +
{{CollapsibleYouTube|CSE 231s Lecture: Windbags|<youtube>orPUQqfOAJw</youtube>}}
  
 
=Code To Implement=
 
=Code To Implement=
 
==WindbagMapper==
 
==WindbagMapper==
−
{{CodeToImplement|WindbagMapper|map|mapreduce.apps.windbag.studio}}
+
{{CodeToImplement|WindbagMapper|windbagLinePredicate<br/>map|mapreduce.apps.windbag.exercise}}
 +
 
 +
===constructor and instance variable===
 +
{{Sequential|public WindbagMapper(IntPredicate windbagLinePredicate)}}
 +
 
 +
Hang onto the value of the windbagLinePredicate parameter as an instance variable for later use.
 +
 
 +
===windbagLinePredicate===
 +
This method exists largely as a reminder to use the windbagLinePredicate passed to the constructor.
 +
===map===
  
−
Note: the WindBagMapper is constructed with an [https://docs.oracle.com/javase/8/docs/api/java/util/function/IntPredicate.html IntPredicate].  This IntPredicate's test method should be invoked to determine if a [https://www.cse.wustl.edu/~dennis.cosgrove/courses/cse231/s20/apidocs/mapreduce/apps/play/core/PlayLine.html PlayLine]'s word count qualifies it as a windbag line. You can investigate the PlayLine class to see what information it contains.
+
Note: the WindBagMapper is constructed with an [https://docs.oracle.com/javase/8/docs/api/java/util/function/IntPredicate.html IntPredicate].  This IntPredicate's test method should be invoked to determine if a [https://www.cse.wustl.edu/~dennis.cosgrove/courses/cse231/current/apidocs/mapreduce/apps/play/core/PlayLine.html PlayLine]'s word count qualifies it as a windbag line. You can investigate the PlayLine class to see what information it contains.
  
−
{{Sequential|public void map(PlayLine item, BiConsumer<PlayRole, Boolean> keyValuePairConsumer)}}
+
{{Sequential|public List<Map.Entry<PlayRole, Boolean>> map(PlayLine item)}}
  
 
==TrueCountClassicReducer==
 
==TrueCountClassicReducer==
Line 27: Line 39:
 
{{Sequential|public Integer reduce(List<Boolean> container)}}
 
{{Sequential|public Integer reduce(List<Boolean> container)}}
  
−
{{Warning|This class inherits from ClassicReducer.  Be sure to have completed that studio first.}}
+
{{Warning|This class inherits from ClassicReducer.  Be sure to have completed that exercise first.}}
  
 
==TruePortionClassicReducer==
 
==TruePortionClassicReducer==
Line 41: Line 53:
 
{{Sequential|public Double reduce(List<Boolean> container)}}
 
{{Sequential|public Double reduce(List<Boolean> container)}}
  
−
{{Warning|This class inherits from ClassicReducer.  Be sure to have completed that studio first.}}
+
{{Warning|This class inherits from ClassicReducer.  Be sure to have completed that exercise first.}}
−
 
 
−
=Testing Your Solution=
 
−
==Correctness==
 
−
{{TestSuite|WindbagStudioTestSuite|mapreduce}}
 
−
 
 
−
==Output==
 
−
 
 
−
{{Performance|WindbagApp|mapreduce.apps.windbag.client}}
 
  
 +
=Client=
 +
{{Client|WindbagApp|mapreduce.apps.windbag.client}}
  
−
<pre>
+
{{CollapsibleConsole|HAMLET|<pre style="border: 0px; background: #000; color:#fff;">
 
           COUNT
 
           COUNT
 
===========================
 
===========================
Line 87: Line 93:
 
  ROSENCRANTZ: 0.022222
 
  ROSENCRANTZ: 0.022222
 
     LAERTES: 0.016129
 
     LAERTES: 0.016129
−
       QUEEN: 0.014493
+
       QUEEN: 0.014493</pre>}}
−
</pre>
+
 
 +
{{CollapsibleConsole|THE_IMPORTANCE_OF_BEING_EARNEST|<pre style="border: 0px; background: #000; color:#fff;">          COUNT
 +
===========================
 +
LADY BRACKNELL:  7
 +
          JACK:  3
 +
      ALGERNON:  2
 +
    GWENDOLEN:  2
 +
    MISS PRISM:  1
 +
      CHASUBLE:  1
 +
 
 +
          PORTION
 +
===========================
 +
LADY BRACKNELL: 0.083333
 +
    MISS PRISM: 0.024390
 +
      CHASUBLE: 0.023810
 +
    GWENDOLEN: 0.019802
 +
          JACK: 0.013825
 +
      ALGERNON: 0.009950</pre>}}
 +
 
 +
=Testing Your Solution=
 +
{{TestSuite|_WindbagTestSuite|mapreduce.apps.windbag.exercise}}
 +
 
 +
=Pledge, Acknowledgments, Citations=
 +
{{Pledge|mapreduce-windbag-app}}

Latest revision as of 09:20, 7 March 2024

Motivation

A simple Mapper and two simple ClassicReducers reinforce our understanding of how pieces of the MapReduce puzzle can be reused. Further, we will reduce the value type of List<Boolean> to both an Integer and a Double in separate Reducers.

Background

A windbag is an exhaustively talkative person. Act I Scene 3 of Hamlet has a famous example.

In this assignment we will try to determine who is the biggest windbag in a particular play with Hamlet and The Importance of Being Earnest serving as two pieces of data.

Lecture

Video: CSE 231s Lecture: Windbags  

Code To Implement

WindbagMapper

class: WindbagMapper.java Java.png
methods: windbagLinePredicate
map
package: mapreduce.apps.windbag.exercise
source folder: student/src/main/java

constructor and instance variable

method: public WindbagMapper(IntPredicate windbagLinePredicate) Sequential.svg (sequential implementation only)

Hang onto the value of the windbagLinePredicate parameter as an instance variable for later use.

windbagLinePredicate

This method exists largely as a reminder to use the windbagLinePredicate passed to the constructor.

map

Note: the WindBagMapper is constructed with an IntPredicate. This IntPredicate's test method should be invoked to determine if a PlayLine's word count qualifies it as a windbag line. You can investigate the PlayLine class to see what information it contains.

method: public List<Map.Entry<PlayRole, Boolean>> map(PlayLine item) Sequential.svg (sequential implementation only)

TrueCountClassicReducer

This Collector reduces to the number of emitted trues.

For example, if a total of eight (8) values were emitted for a particular key and three (3) of them were true:

[false, true, false, false, true, true, false, false]

then the reduced value would be 3

class: TrueCountClassicReducer.java Java.png
methods: reduce
package: mapreduce.apps.windbag.exercise
source folder: student/src/main/java

method: public Integer reduce(List<Boolean> container) Sequential.svg (sequential implementation only)

Attention niels epting.svg Warning:This class inherits from ClassicReducer. Be sure to have completed that exercise first.

TruePortionClassicReducer

This Collector reduces to the portion of emitted trues.

For example, if a total of eight (8) values were emitted for a particular key and three (3) of them were true:

[false, true, false, false, true, true, false, false]

then the reduced value would be 0.375

class: TruePortionClassicReducer.java Java.png
methods: reduce
package: mapreduce.apps.windbag.exercise
source folder: student/src/main/java

method: public Double reduce(List<Boolean> container) Sequential.svg (sequential implementation only)

Attention niels epting.svg Warning:This class inherits from ClassicReducer. Be sure to have completed that exercise first.

Client

class: WindbagApp.java CLIENT
package: mapreduce.apps.windbag.client
source folder: student/src//java
HAMLET   
           COUNT
===========================
      HAMLET: 27
        KING: 14
    POLONIUS:  7
     HORATIO:  5
     OPHELIA:  2
       GHOST:  2
FIRST PLAYER:  2
     LAERTES:  1
 ROSENCRANTZ:  1
   VOLTEMAND:  1
 PLAYER KING:  1
   GENTLEMAN:  1
PLAYER QUEEN:  1
       QUEEN:  1



          PORTION
===========================
   VOLTEMAND: 1.000000
   GENTLEMAN: 0.333333
 PLAYER KING: 0.250000
FIRST PLAYER: 0.250000
PLAYER QUEEN: 0.200000
       GHOST: 0.142857
        KING: 0.137255
    POLONIUS: 0.081395
      HAMLET: 0.075419
     HORATIO: 0.046729
     OPHELIA: 0.034483
 ROSENCRANTZ: 0.022222
     LAERTES: 0.016129
       QUEEN: 0.014493
THE_IMPORTANCE_OF_BEING_EARNEST   
           COUNT
===========================
LADY BRACKNELL:  7
          JACK:  3
      ALGERNON:  2
     GWENDOLEN:  2
    MISS PRISM:  1
      CHASUBLE:  1

          PORTION
===========================
LADY BRACKNELL: 0.083333
    MISS PRISM: 0.024390
      CHASUBLE: 0.023810
     GWENDOLEN: 0.019802
          JACK: 0.013825
      ALGERNON: 0.009950

Testing Your Solution

class: _WindbagTestSuite.java Junit.png
package: mapreduce.apps.windbag.exercise
source folder: testing/src/test/java

Pledge, Acknowledgments, Citations

file: mapreduce-windbag-app-pledge-acknowledgments-citations.txt

More info about the Honor Pledge

Retrieved from "http://classes.engineering.wustl.edu/cse231/core/index.php?title=Windbag_MapReduce_Application&oldid=5162"