AWS Lambda CPU Profiling (Java)

Performance profiling is essential for optimizing and fixing an application’s resource consumption and latency. A performance issue without an execution profile is like an error without a stack trace. It will lead to a lot of manual work to get to the root cause.

Profiling cloud applications and functions such as AWS Lambda requires special profiling tools designed for the cloud environments (specifically FaaS) due to their restrictions on background tasks, system calls, incoming connections, etc. On top of that, it is quite unrealistic to simulate a cloud environment with all its data, traffic and configuration locally.

Adding the StackImpact profiler agent to the Lambda function

StackImpact cloud profiler was specifically designed for cloud production environments. Unlike traditional profilers, which usually only run locally, the StackImpact profiler runs inside of the cloud applications and completely automates the burdensome profiling process.

The following simple AWS Lambda Java function simulates some CPU work. Adding the StackImpact Java agent is only a few lines of code. More details can found in the StackImpact Java package GitHub page or in the documentation.

package examples;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.BufferedReader;
import java.io.Writer;
import java.util.Random;

import com.amazonaws.services.lambda.runtime.RequestStreamHandler;
import com.amazonaws.services.lambda.runtime.Context; 


import org.json.simple.JSONObject;
import org.json.simple.JSONArray;
import org.json.simple.parser.ParseException;
import org.json.simple.parser.JSONParser;

import com.stackimpact.agent.StackImpact;
import com.stackimpact.agent.ProfileSpan;


public class Hello implements RequestStreamHandler {
    JSONParser parser = new JSONParser();


    // initialize StackImpact agent
    static {
        StackImpact.setAutoProfilingMode(false);
        StackImpact.start("agent key here", "LambdaJavaApp");
    }

    
    public void handleRequest(InputStream inputStream, OutputStream outputStream, Context context) throws IOException {
        ProfileSpan span = StackImpact.profile();

        // simulate CPU work
        try {
            Random rand = new Random();
            for (int i = 0; i < 20 * 1000000; i++) {
                rand.nextInt(100000);
            }
        }
        catch(Exception ex) {
        }
        // end CPU work

        BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
        JSONObject responseJson = new JSONObject();
        String responseCode = "200";

        try {
            JSONObject event = (JSONObject)parser.parse(reader);

            JSONObject responseBody = new JSONObject();
            responseBody.put("input", event.toJSONString());
            responseBody.put("message", "Hello");

            JSONObject headerJson = new JSONObject();
            headerJson.put("x-custom-header", "my custom header value");

            responseJson.put("isBase64Encoded", false);
            responseJson.put("statusCode", responseCode);
            responseJson.put("headers", headerJson);
            responseJson.put("body", responseBody.toString());  

        } catch(ParseException pex) {
            responseJson.put("statusCode", "400");
            responseJson.put("exception", pex);
        }

        OutputStreamWriter writer = new OutputStreamWriter(outputStream, "UTF-8");
        writer.write(responseJson.toJSONString());  
        writer.close();

        span.stop();
    }
}

You can get the agent key by signing up for a free trial account. The auto profiling is disabled with the StackImpact.setAutoProfiling(false) agent startup option, since the agent cannot initiate profiling automatically. This is because of restrictions on background tasks, more precisely, the Lambda process “freezes” when the handler is inactive. This is also why the StackImpact.profile() method should be used.

Locating CPU hot spots

After generating requests against this Lambda function for some period of time, the CPU hot spots can be easily located in the reported profiles in the StackImpact Dashboard’s Hot spots / CPU section.

Screenshot

Continuous performance profiling

One of the important benefits of continuously profiling an application is that the profiles can be historically analyzed and compared. Unlike one-time call graphs, a call graph history per process and application allows a much deeper understanding of application execution. For example, the root cause of the performance regression in a new version of the function can be easily identified.

Since there can be many instances of Lambda containers, only a small subset will have active profiling agents.

See full documentation for more details.