MapReduce: Functional programming with Node.JS

 

In the context of big data, it is often normal we have arrays with thousand entries. And operations to filter, transform (map) or calculate statistics indicators (reduce) had to be often performed.

However, without techniques of functional programming, one of few alternatives is to program that using loops, such as for statements, while  statements, or do-while statements.

However with map reduce techniques we can focus just in specific functions to be programmed, according with the business rule, and all the complexity to handle big datasets, such as memory consumption and another optimizations are performed by the own technology.

The first function we are going to study, is the filter function. Basically it has to receive a function as parameter. This function receives as parameter a current element (it will be executed for each array element) and return a Boolean value. If true the value is maintained in the output value, and if false it will not be presented in the output array. The filter function will produces a new array with filtered elements.

//filter

console.log("---Filter Method---");

let newArrayFiltered=arrayB.filter((el, index, array)=>{

    if(typeof(el)!= "undefined"){

        return true;

    }else{

        return false;

    }

});

console.log("Filtered", newArrayFiltered);

 

The second function is the map. It also receives as a parameter another function, that receives as parameter the current element, and produces as output he same element, but transformed. It means a new value to be placed in the same index as the current value, but transformed. The output of the map function is an array with the same length as the original array, but with elements transformed.

 

 

 

//map

console.log("---Map Method---");

let newArrayMapped=newArrayFiltered.map((el, index, array)=>{

    return el*2;//double array values

});

console.log("Mapped", newArrayMapped);

 

The last function we are going to study is the reduce. It basically will transform all values of an array in a new value, normally a scalar value, it could be a sum, mean, median standard deviation, and so on.

//reduce

console.log("---Reduce Method---");

let reducedValueSum=newArrayMapped.reduce((prevVal, el)=>{

    return prevVal+el;//sum all elements

});

console.log("Reduced - SUM", reducedValueSum);

 

 

The source code is available in github:

 https://github.com/rafaelqg/code/blob/main/arrays_map_reduce.js

You can watch a video class about these functions in this video below:





Comments

Post a Comment

Popular posts from this blog

Dart/Flutter: exception handling

Image
The exceptions are triggered every time an error happens during the software execution. In case exceptions are not handled, the software will abort, and the program execution will ends. To make the software with capability to handle its errors and give to the users a proper feedback and possibility to correct some input or try again, the exceptions can be handled. To perform exception handling using Dart, which is the programming language adopted by Flutter technology, you may use the try / on statement was we have in the example below: int x = 12 ; int y = 0 ; int res ; print( "1) try / on statement" ); try {   res = x ~/ y ; // ~/ => truncating division operator, always returns a int value. } on IntegerDivisionByZeroException {   print( 'Exception:: Cannot divide by zero' ); } Another possibility is to have the same structure but with the addition of the catch keyword. It makes possible to have access to the exception object and to read data from t...
Read more

Dart: implementing Object Oriented Programming (OOP)

Image
The object oriented programming provides several benefits for system design. It makes the code easier to maintain, easier to extends, and also implements several well-know design patterns. Using the Dart programing language classes can be simple defined utilizing the class keyword, followed by the class name, and then delimitated the class body with the key brackets {}, like in the example below, presenting one class with two methods. class ProgrammingLanguage {   void compile (){     print( "Compiling..." );   }   void execute (){     print ( "Executing..." );   } } When we need to add new features to this class, it is possible to use the inherence concept, defining a new class that will receive all implementation this ProgrammingLanguage class already has done. And this inherence is done using the extends keywork, like in the example below, which the Dart class has the compile() and the execute() methos from its superclass...
Read more

Android – SearchView: Adding a Search component direct on your APP menu.

Image
  A SearchView is View (GUI Component) to be displayed in the toolbar, which user can enter a text for searching anything within your APP. Of course, programmatically we can Interact with SeachView to identify the its interaction with the user and the inputted values. However the whole search behavior has to be implemented by each application.     In this tutorial we are going to see how to include a SearchView in your APP toolbar, and how to handle its events.     The SearchView is defined in any layout for menus, and its is like a normal menu item. However it has different behavior because of the attributes   app :actionViewClass and app :showAsAction as we can see in the code below:   <? xml version ="1.0" encoding ="utf-8" ?> < menu xmlns: app ="http://schemas.android.com/apk/res-auto"     xmlns: android ="http://schemas.android.com/apk/res/android" >     < item    ...
Read more