Difference between revisions of "Windbag MapReduce Application"

From CSE231 Wiki
Jump to navigation Jump to search
 
(2 intermediate revisions by the same user not shown)
Line 6: Line 6:
  
 
In this assignment we will try to determine who is the biggest windbag in a particular play with [https://en.wikipedia.org/wiki/Hamlet Hamlet] and [https://en.wikipedia.org/wiki/The_Importance_of_Being_Earnest The Importance of Being Earnest] serving as two pieces of data.
 
In this assignment we will try to determine who is the biggest windbag in a particular play with [https://en.wikipedia.org/wiki/Hamlet Hamlet] and [https://en.wikipedia.org/wiki/The_Importance_of_Being_Earnest The Importance of Being Earnest] serving as two pieces of data.
 +
 +
=Lecture=
 +
{{CollapsibleYouTube|CSE 231s Lecture: Windbags|<youtube>orPUQqfOAJw</youtube>}}
  
 
=Code To Implement=
 
=Code To Implement=
 
==WindbagMapper==
 
==WindbagMapper==
−
{{CodeToImplement|WindbagMapper|windbagLinePredicate<br/>map|mapreduce.apps.windbag.studio}}
+
{{CodeToImplement|WindbagMapper|windbagLinePredicate<br/>map|mapreduce.apps.windbag.exercise}}
  
 
===constructor and instance variable===
 
===constructor and instance variable===
Line 20: Line 23:
 
===map===
 
===map===
  
−
Note: the WindBagMapper is constructed with an [https://docs.oracle.com/javase/8/docs/api/java/util/function/IntPredicate.html IntPredicate].  This IntPredicate's test method should be invoked to determine if a [https://www.cse.wustl.edu/~dennis.cosgrove/courses/cse231/s20/apidocs/mapreduce/apps/play/core/PlayLine.html PlayLine]'s word count qualifies it as a windbag line. You can investigate the PlayLine class to see what information it contains.
+
Note: the WindBagMapper is constructed with an [https://docs.oracle.com/javase/8/docs/api/java/util/function/IntPredicate.html IntPredicate].  This IntPredicate's test method should be invoked to determine if a [https://www.cse.wustl.edu/~dennis.cosgrove/courses/cse231/current/apidocs/mapreduce/apps/play/core/PlayLine.html PlayLine]'s word count qualifies it as a windbag line. You can investigate the PlayLine class to see what information it contains.
  
 
{{Sequential|public List<Map.Entry<PlayRole, Boolean>> map(PlayLine item)}}
 
{{Sequential|public List<Map.Entry<PlayRole, Boolean>> map(PlayLine item)}}
Line 36: Line 39:
 
{{Sequential|public Integer reduce(List<Boolean> container)}}
 
{{Sequential|public Integer reduce(List<Boolean> container)}}
  
−
{{Warning|This class inherits from ClassicReducer.  Be sure to have completed that studio first.}}
+
{{Warning|This class inherits from ClassicReducer.  Be sure to have completed that exercise first.}}
  
 
==TruePortionClassicReducer==
 
==TruePortionClassicReducer==
Line 50: Line 53:
 
{{Sequential|public Double reduce(List<Boolean> container)}}
 
{{Sequential|public Double reduce(List<Boolean> container)}}
  
−
{{Warning|This class inherits from ClassicReducer.  Be sure to have completed that studio first.}}
+
{{Warning|This class inherits from ClassicReducer.  Be sure to have completed that exercise first.}}
−
 
 
  
 
=Client=
 
=Client=

Latest revision as of 09:20, 7 March 2024

Motivation

A simple Mapper and two simple ClassicReducers reinforce our understanding of how pieces of the MapReduce puzzle can be reused. Further, we will reduce the value type of List<Boolean> to both an Integer and a Double in separate Reducers.

Background

A windbag is an exhaustively talkative person. Act I Scene 3 of Hamlet has a famous example.

In this assignment we will try to determine who is the biggest windbag in a particular play with Hamlet and The Importance of Being Earnest serving as two pieces of data.

Lecture

Video: CSE 231s Lecture: Windbags  

Code To Implement

WindbagMapper

class: WindbagMapper.java Java.png
methods: windbagLinePredicate
map
package: mapreduce.apps.windbag.exercise
source folder: student/src/main/java

constructor and instance variable

method: public WindbagMapper(IntPredicate windbagLinePredicate) Sequential.svg (sequential implementation only)

Hang onto the value of the windbagLinePredicate parameter as an instance variable for later use.

windbagLinePredicate

This method exists largely as a reminder to use the windbagLinePredicate passed to the constructor.

map

Note: the WindBagMapper is constructed with an IntPredicate. This IntPredicate's test method should be invoked to determine if a PlayLine's word count qualifies it as a windbag line. You can investigate the PlayLine class to see what information it contains.

method: public List<Map.Entry<PlayRole, Boolean>> map(PlayLine item) Sequential.svg (sequential implementation only)

TrueCountClassicReducer

This Collector reduces to the number of emitted trues.

For example, if a total of eight (8) values were emitted for a particular key and three (3) of them were true:

[false, true, false, false, true, true, false, false]

then the reduced value would be 3

class: TrueCountClassicReducer.java Java.png
methods: reduce
package: mapreduce.apps.windbag.exercise
source folder: student/src/main/java

method: public Integer reduce(List<Boolean> container) Sequential.svg (sequential implementation only)

Attention niels epting.svg Warning:This class inherits from ClassicReducer. Be sure to have completed that exercise first.

TruePortionClassicReducer

This Collector reduces to the portion of emitted trues.

For example, if a total of eight (8) values were emitted for a particular key and three (3) of them were true:

[false, true, false, false, true, true, false, false]

then the reduced value would be 0.375

class: TruePortionClassicReducer.java Java.png
methods: reduce
package: mapreduce.apps.windbag.exercise
source folder: student/src/main/java

method: public Double reduce(List<Boolean> container) Sequential.svg (sequential implementation only)

Attention niels epting.svg Warning:This class inherits from ClassicReducer. Be sure to have completed that exercise first.

Client

class: WindbagApp.java CLIENT
package: mapreduce.apps.windbag.client
source folder: student/src//java
HAMLET   
           COUNT
===========================
      HAMLET: 27
        KING: 14
    POLONIUS:  7
     HORATIO:  5
     OPHELIA:  2
       GHOST:  2
FIRST PLAYER:  2
     LAERTES:  1
 ROSENCRANTZ:  1
   VOLTEMAND:  1
 PLAYER KING:  1
   GENTLEMAN:  1
PLAYER QUEEN:  1
       QUEEN:  1



          PORTION
===========================
   VOLTEMAND: 1.000000
   GENTLEMAN: 0.333333
 PLAYER KING: 0.250000
FIRST PLAYER: 0.250000
PLAYER QUEEN: 0.200000
       GHOST: 0.142857
        KING: 0.137255
    POLONIUS: 0.081395
      HAMLET: 0.075419
     HORATIO: 0.046729
     OPHELIA: 0.034483
 ROSENCRANTZ: 0.022222
     LAERTES: 0.016129
       QUEEN: 0.014493
THE_IMPORTANCE_OF_BEING_EARNEST   
           COUNT
===========================
LADY BRACKNELL:  7
          JACK:  3
      ALGERNON:  2
     GWENDOLEN:  2
    MISS PRISM:  1
      CHASUBLE:  1

          PORTION
===========================
LADY BRACKNELL: 0.083333
    MISS PRISM: 0.024390
      CHASUBLE: 0.023810
     GWENDOLEN: 0.019802
          JACK: 0.013825
      ALGERNON: 0.009950

Testing Your Solution

class: _WindbagTestSuite.java Junit.png
package: mapreduce.apps.windbag.exercise
source folder: testing/src/test/java

Pledge, Acknowledgments, Citations

file: mapreduce-windbag-app-pledge-acknowledgments-citations.txt

More info about the Honor Pledge

Retrieved from "http://classes.engineering.wustl.edu/cse231/core/index.php?title=Windbag_MapReduce_Application&oldid=5162"