Apache cochon - Convesion de ruche à PigLatin - Stack Overflow

I am trying to convert the below Hive statement to Pig:

max(substr(case when url like 'http:%' then '' else url end,1,50))

My pig statement for the above is:

url_group = GROUP data by (uid);
max_substr_url= FOREACH url_group generate SUBSTRING(MAX(((Coalesce(data.url) matches '.*http:%.*') ? '' : Coalesce(data.url))), 0, 49);

For some of the data, the url can be null. So I have written a pig UDF called Coalesce(String) which returns an empty string if the data is either null or empty. If the data is not null or not empty it returns the string back.

The above pig statement is giving me lot of trouble and tried n different options/ways but nothing worked. Anyone got any ideas on how to implement this? Please help me.

Thanks in advance

You are going to want to use a nested FOREACH so that you can do the substring transformation on each tuple in the data bag then take the MAX of the transformed bag.

A = GROUP data by (uid);
B = FOREACH url_group {
    -- MAX needs a one column bag
    transformed = FOREACH data 
                  GENERATE SUBSTRING((Coalesce(url) matches '.*http:.*' ? '' : Coalesce(url)), 0, 49);
    GENERATE group AS uid, MAX(transformed) ;
}

I am trying to convert the below Hive statement to Pig:

max(substr(case when url like 'http:%' then '' else url end,1,50))

My pig statement for the above is:

url_group = GROUP data by (uid);
max_substr_url= FOREACH url_group generate SUBSTRING(MAX(((Coalesce(data.url) matches '.*http:%.*') ? '' : Coalesce(data.url))), 0, 49);

For some of the data, the url can be null. So I have written a pig UDF called Coalesce(String) which returns an empty string if the data is either null or empty. If the data is not null or not empty it returns the string back.

The above pig statement is giving me lot of trouble and tried n different options/ways but nothing worked. Anyone got any ideas on how to implement this? Please help me.

Thanks in advance

You are going to want to use a nested FOREACH so that you can do the substring transformation on each tuple in the data bag then take the MAX of the transformed bag.

A = GROUP data by (uid);
B = FOREACH url_group {
    -- MAX needs a one column bag
    transformed = FOREACH data 
                  GENERATE SUBSTRING((Coalesce(url) matches '.*http:.*' ? '' : Coalesce(url)), 0, 49);
    GENERATE group AS uid, MAX(transformed) ;
}

Source