Java - How to Send POST Requests With Apache HttpClient & OKHttp
In this guide for The Java Web Scraping Playbook, we will look at how to make POST requests with two popular Java libraries:
We will walk you through the most common ways of sending POST requests using both libraries:
- POST JSON Data
- POST Form Data
Let's begin...
Need help scraping the web?
Then check out ScrapeOps, the complete toolkit for web scraping.
POST JSON Data
A common scenario for using POST requests is to send JSON data to an API endpoint, etc.
We will show how to do this with both Apache HttpClient and OKHttp.
- Apache HttpClient
- OKHttp
Before you can make POST requests with Java Apache HttpClient, you need to import all the required classes from this library.
import org.apache.hc.client5.http.async.methods.SimpleHttpRequest;
import org.apache.hc.client5.http.async.methods.SimpleHttpResponse;
import org.apache.hc.client5.http.async.methods.SimpleRequestBuilder;
import org.apache.hc.client5.http.impl.async.CloseableHttpAsyncClient;
import org.apache.hc.client5.http.impl.async.HttpAsyncClients;
import org.apache.hc.core5.http.ContentType;
import java.util.concurrent.Future;
Now inside the main code, initialize requestUrl and jsonData variables.
String requestUrl = "https://httpbin.org/post";
String jsonData = "{ \"key\": \"value\" }";
After that create an instance of CloseableHttpClient using HttpsClients.createDefault method and set it to client variable. Call client.start method to initialize internal resources so that client is ready to make asynchronous http requests.
CloseableHttpAsyncClient client = HttpAsyncClients.createDefault();
client.start();
Next use SimpleRequestBuilder.post(requestUrl) to configure post request url. Then configure request body and its content-type by calling setBody method with jsonData and ContentType.APPLICATION_JSON respectively. To actually create your request object based on the supplied configurations, simply call build method.
SimpleHttpRequest request = SimpleRequestBuilder.post(requestUrl)
.setBody(jsonData, ContentType.APPLICATION_JSON)
.build();
Finally to send the post request, just use client.execute method.
Because client.execute is an asynchronous function, it doesn't immediately return response. It instead returns future object which is an instance of java Future<SimpleHttpResponse>. So you need to wait for the server to return response by calling future.get method.
Future<SimpleHttpResponse> future = client.execute(request, null);
SimpleHttpResponse response = future.get();
System.out.println("Response body: " + response.getBodyText());
To wrap up this section, here's the entire code for making POST requests with Java Apache HttpClient:
import org.apache.hc.client5.http.async.methods.SimpleHttpRequest;
import org.apache.hc.client5.http.async.methods.SimpleHttpResponse;
import org.apache.hc.client5.http.async.methods.SimpleRequestBuilder;
import org.apache.hc.client5.http.impl.async.CloseableHttpAsyncClient;
import org.apache.hc.client5.http.impl.async.HttpAsyncClients;
import org.apache.hc.core5.http.ContentType;
import java.util.concurrent.Future;
public class JsonPostRequest {
public static void main(String[] args) throws Exception {
String requestUrl = "https://httpbin.org/post";
String jsonData = "{\"key\": \"value\"}"; // replace with your JSON data
CloseableHttpAsyncClient client = HttpAsyncClients.createDefault();
client.start();
try {
SimpleHttpRequest request = SimpleRequestBuilder.post(requestUrl)
.setBody(jsonData, ContentType.APPLICATION_JSON)
.build();
Future<SimpleHttpResponse> future = client.execute(request, null);
SimpleHttpResponse response = future.get();
System.out.println("Response body: " + response.getBodyText());
} finally {
client.close();
}
}
}
Note that we call client.close method after our code finishes running (inside finally block). By doing so, we release system resources allocated by the client and prevent potential memory leaks.
Before you can make POST requests with Java OKHttp Library, you need to import all the required classes from this library.
import java.util.concurrent.TimeUnit;
import okhttp3.MediaType;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;
Note we are importing our classes from okhttp3 package even though we're using v4 of the library. That's because the same package naming from version 3.x has continued into version 4.x.
Now inside the main code, initialize requestUrl and jsonData variables.
String requestUrl = "https://httpbin.org/post";
String jsonData = "{ \"key\": \"value\" }";
After that use OkHttpClient.Builder constructor to create a new OkHttpClient builder. Call readTimeout method on the builder to set response timeout of 30 seconds. Finally chain call build method to build an instance of OkHttpClient named client. It's the client object that allows us to send requests to the server and receive responses.
OkHttpClient client = new OkHttpClient.Builder()
.readTimeout(30, TimeUnit.SECONDS)
.build();
If we didn't need to customize request timeout on client, we could simplify the above code to the following:
OkHttpClient client = new OkHttpClient();
You have to provide your request Content-Type in the form of MediaType object. Use MediaType.get factory method and provide "application/json" as an argument to create your contentType. Then initialize your request body by calling RequestBody.create method with jsonData string and contentType object.
MediaType contentType = MediaType.get("application/json");
RequestBody body = RequestBody.create(jsonData, contentType);
Next use Request.Builder constructor to create a new OkHttp Request builder. Then configure requestUrl and post body by calling url and post methods respectively. To actually create your request object based on the supplied configurations, simply call build method.
Request request = new Request.Builder()
.url(requestUrl)
.post(body)
.build();
Finally to send the post request, just call client.newCall(request).execute method.
Response response = client.newCall(request).execute();
System.out.println("Response body: " + response.body().string());
Here's the entire code for making POST requests with Java OKHttp Library:
import java.util.concurrent.TimeUnit;
import okhttp3.MediaType;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;
public class FormDataPostRequest {
public static void main(String[] args) throws Exception {
String requestUrl = "https://httpbin.org/post";
String jsonData = "{ \"key\": \"value\" }";
OkHttpClient client = new OkHttpClient.Builder()
.readTimeout(30, TimeUnit.SECONDS)
.build();
MediaType mediaType = MediaType.get("application/json");
RequestBody body = RequestBody.create(jsonData, mediaType);
Request request = new Request.Builder()
.url(requestUrl)
.post(body)
.build();
Response response = client.newCall(request).execute();
System.out.println("Response body: " + response.body().string());
}
}
POST Form Data
Another common use case is to send form-encoded data in a POST request. Let's see how to do this with both libraries as well.
- Apache HttpClient
- OKHttp
In this section, we'll see how we can make post request with the following formData.
String formData = "key1=value1&key2=value2";
To make form data POST requests with Apache HttpClient, simply call setBody method of request builder with formData as the first argument. The second argument is content type, which we set to ContentType.APPLICATION_FORM_URLENCODED so that the formData will indeed be picked up as form data.
SimpleHttpRequest request = SimpleRequestBuilder.post(requestUrl)
.setBody(formData, ContentType.APPLICATION_FORM_URLENCODED)
.build();
Here's the full code sample:
// imports have been left out here for sake of brevity
public class FormDataPostRequest {
public static void main(String[] args) throws Exception {
String requestUrl = "https://httpbin.org/post";
String formData = "key1=value1&key2=value2"; // replace with your form data
CloseableHttpAsyncClient client = HttpAsyncClients.createDefault();
client.start();
try {
SimpleHttpRequest request = SimpleRequestBuilder.post(requestUrl)
.setBody(formData, ContentType.APPLICATION_FORM_URLENCODED)
.build();
Future<SimpleHttpResponse> future = client.execute(request, null);
SimpleHttpResponse response = future.get();
System.out.println("Response body: " + response.getBodyText());
} finally {
client.close();
}
}
}
In this section, we'll see how we can make post request with the following formData:
String formData = "key1=value1&key2=value2";
To make form data POST requests with OKHttp, initialize contentType object by calling MediaType.get method with "application/x-www-form-urlencoded" so that formData will indeed be picked up as form data. Then create request body using RequestBody.create(formData, contentType) and pass it to post method of request builder.
MediaType contentType = MediaType.get("application/x-www-form-urlencoded");
RequestBody body = RequestBody.create(formData, contentType);
Request request = new Request.Builder()
.url(requestUrl)
.post(body)
.build();
Here's the full code sample:
// imports have been left out here
public class FormDataPostRequest {
public static void main(String[] args) throws Exception {
String requestUrl = "https://httpbin.org/post";
String formData = "key1=value1&key2=value2";
OkHttpClient client = new OkHttpClient.Builder()
.readTimeout(30, TimeUnit.SECONDS)
.build();
MediaType contentType = MediaType.get("application/x-www-form-urlencoded");
RequestBody body = RequestBody.create(formData, contentType);
Request request = new Request.Builder()
.url(requestUrl)
.post(body)
.build();
Response response = client.newCall(request).execute();
System.out.println("Response body: " + response.body().string());
}
}
More Web Scraping Tutorials
So that's how you can send POST requests using Java Apache HttpClient and Java OKHttp Library. If you'd like to learn more about web scraping or how to handle requests in various libraries, check out some of the additional resources below: