Export Users using the User Export API

Export users from your project into a CSV or JSON file

The Export User API offers a means for exporting user data such as user ID, email, phone number, etc from your Authgear project into a CSV or ndjson file.

In this guide, you'll learn how to use the User Export API.

User Export API

The User Export API allows developers to bulk export users into a file.

The export process is asynchronous. That is, the process runs in the background. Hence, you will need to initiate an export task in one endpoint call and then, make an additional call to another endpoint to get the status of the export task.

Authentication

To use the Export User API, your application must be authenticated using an Admin API JWT token.

Once you obtain a valid Admin API JWT, you can set it as a Bearer Authorization header to access the Export User API Endpoints. The API will return a "403 Forbidden" error if an invalid JWT is used.

The following are the two endpoints for the User Export API:

Initiate Export Task

POST /_api/admin/users/export

Use this endpoint to create a new user export task.

Headers

Name	Value
Content-Type	application/json
Authorization	Bearer <Admin API JWT Token>
Host	<Your Authgear Project domain>

Body

The Initiate Export endpoint accepts JSON input via an HTTP(S) request body. The following is an example of the input:

{
  "format": "csv",
  "csv": {
    "fields": [
      {
        "pointer": "/sub",
        "field_name": "user_id"
      },
      {
        "pointer": "/email"
      }
    ]
  }
}

• The format field is where you specify the format of the export file. The value can be csv or ndjson.

• csv: use this field when format is set to csv. The value is an object with a fields property.

• csv.fields: you can use this field to list all the user attributes you want to include as fields in the CSV file. The value should be an array and each item in the array is an object with a pointer and an optional field_name property that describe a user attribute.

User Profile Attribute Pointers

The following is a complete list of supported pointers:

Pointer	Description	Value
/sub	sub is short for "subject" and it is a standard field in O. The value is the user's unique ID generated by Authgear	String
/preferred_username	the user's chosen username.	String
/email	the email address associated with the current user account	String
/phone_number	a phone number associated with the current user's account	String
/email_verified	contains a boolean value that represents the current status of the user's email verification. The value can be either true or false.	Boolean
/phone_number_verified	returns true if the associated phone number is verified and false if it's not.	Boolean
/name	the user's display name.	String
/given_name	the user's given name	String
/middle_name	the user's middle name	String
/nickname	a nickname set by the user.	String
/profile	link to the user's profile	String
/picture	contains the URL to the user's profile photo.	String
/website	the user's website. That is, the value a user sets in the website attribute for their profile.	String
/gender	holds the value set for the current users' gender (male, female, or some other string).	String
/birthdate	the user's birthday set in their profile's standard attribute.	String
/zoneinfo	the user's timezone. For example Asia/Hong_Kong, Europe/London.	String
/locale	display language set by user. For example en for English.	String
/address/street_address	the value the user sets for the street address field in their address standard attribute.	String
/address/locality	the value the user sets for the city (locality) field in their address standard attribute.	String
/address/region	the value the user sets for the "State / Province / Prefecture / Region" field in their address standard attribute.	String
/address/postal_code	the postcode in the user's address standard attribute.	String
/address/country	the value the user sets for the country field in their address standard attribute.	String
/roles	a list of all the roles assigned to the current user.	Array
/groups	a list of all the access management groups the user has been added to.	Array
/disabled	the status of the user's account. The value is false if the account is not disabled and true if it is disabled.	Boolean
/identities	a list of all the identities associated with a user's account. The structure of the value looks like this: [{"claims":{"email":"[e-mail]"},"login_id":{"key":"email","original_value":"[e-mail]","type":"email","value":"[e-mail]"},"type":"login_id"},{"claims":{"phone_number":"[phone]"},"login_id":{"key":"phone","original_value":"[phone]","type":"phone","value":"[phone]"},"type":"login_id"}]	Array
/mfa/emails	a list of email addresses the user has added to receive 2FA login link or OTP.	Array
/mfa/phone_numbers	a list of phone numbers the user has added to receive OTP via SMS or WhatsApp.	Array
/mfa/totps	a list of JSON objects that contain details such as secret and uri for TOTP. The following an example of the struture of the data in this field: [ { "secret":"ABCD1234EF...", "uri":"otpauth://totp/[e-mail]?algorithm=SHA1\u0026digits=6\u0026issuer=https%3A%2F%2F127.0.0.1\u0026period=30\u0026secret=ABCD1234EF..." } ]	Array
/biometric_count	number of of biometric registered for the user.	Integer
/passkey_count	number of passkey registered for the user.	Integer

If csv.fields is not specified, all the above pointers will be included as fields in the exported CSV file.

The value for pointer for a custom attribute will follow this format:

{"pointer": "/custom_attributes/CUSTOM_ATTRIBUTE_NAME"}

For example, a custom attribute named member_id will be {"pointer": "/custom_attributes/member_id"}.

If field_name is not given, then it is derived from pointer using the following rules:

1. Let parts be the list of the reference tokens in pointer.

2. Join parts with the character ..

For example, following the above roles, the derived field_name for {"pointer": "/address/formatted"} will be address.formatted.

Response

The body of an HTTP(S) request to the Initiate Export endpoint looks like this:

200 OK

{
    "result": {
        "id": "userexport_VWFAACAMB5V0BY1J7KK5NS3GV1TQAACQ",
        "created_at": "2024-10-07T21:06:24.361634372Z",
        "status": "pending",
        "request": {
            "format": "csv",
            "csv": {
                "fields": [
                    {
                        "pointer": "/sub",
                        "field_name": "user_id"
                    },
                    {
                        "pointer": "/email"
                    },
                    {
                        "pointer": "/mfa/emails"
                    }
                ]
            }
        }
    }
}

403 Forbidden

Forbidden

Please check if the JWT in the Authorization header is valid.

• id: the value for this field is the export task ID required in the check status endpoint.

• created_at: contains the date and time an export task was created.

• status: returns pending when the task is created successfully.

Error Response

Description	Name	Reason	Info
When user export is disabled	InternalError	UserExportDisabled	-
When the rate limit exceeded	TooManyRequest	RateLimited	{"bucket_name": "UserExport"}
When there is a running export	TooManyRequest	MaximumConcurrentJobLimitExceeded	-
When the input fails the validation	Invalid	ValidationFailed	The info should contain the JSON schema validation output
When the field names are not unique	Invalid	UserExportNonUniqueFieldNames	Output the full list of "field_names". Like {"field_names": ["sub", "a", "b", "a"]}

Check Status

GET /_api/admin/users/export/{Task ID}

Use this endpoint to query the status of an existing export task. Replace {Task ID} with the task id returned in the response body of the initiate export endpoint.

Headers

Name	Value
Content-Type	application/json
Authorization	Bearer <Admin API JWT Token>
Host	<Your Authgear Project domain>

Response

The following is what the response body of the Check Status endpoint looks like:

200

{
    "result": {
        "id": "userexport_VWFAACAMB5V0BY1J7KK5NS3GV1TQAACQ",
        "created_at": "2024-10-07T21:06:24.361634372Z",
        "completed_at": "2024-10-07T21:06:24.551711713Z",
        "status": "completed",
        "request": {
            "format": "csv",
            "csv": {
                "fields": [
                    {
                        "pointer": "/sub",
                        "field_name": "user_id"
                    },
                    {
                        "pointer": "/email"
                    },
                    {
                        "pointer": "/mfa/emails"
                    }
                ]
            }
        },
        "download_url": "https://storage.googleapis.com/authgear-userexport-staging/example-userexport_userexport_VWFAACAMB5V0BY1J7KK5NS3GV1TQAACQ-20241007210624Z.csv?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=authgear-server%40oursky-kube.iam.gserviceaccount.com%2F20241007%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20241007T210641Z&X-Goog-Expires=59&X-Goog-Signature=abcd12e45&X-Goog-SignedHeaders=host"
    }
}

400

Forbidden

• The status field reports progress on the current export task. If the value is completed, the export is complete, and you can now download the export file.

• download_url: This is a signed URL that you can use to download the export file.

Note: The result from a completed export task will expire after 24 hours. Hence, after 24 hours, you can no longer use the task ID associated with the task to generate a new download link.

Error Response

Description	Name	Reason	Info
When user export is disabled	InternalError	UserExportDisabled	-
When the given ID does not refer to an export	NotFound	TaskNotFound	-

Download Export File

The response of the check status endpoint will include a download_url field that contains a signed URL that you can open to download the CSV or ndjson user export file.

The signed URL is only valid for 60 seconds, and afterwards, you'll have to make another request to the check status endpoint using the task ID to obtain a new URL.

Example: Using the User Export API

The following example shows how to use the User Export API in a Node.js application.

Step 1: Get Admin API JWT

First, you need to get the Admin API JWT that will be used to authenticate requests to the endpoints.

To do that, install JsonWebToken (a Node package for generating JWT) by running the following command:

npm install jsonwebtoken

Now, create a generateJWT() function in your Express app to generate the JWT:

const node_jwt = require('jsonwebtoken');
const fs = require("fs");

function generateJWT() {
    const project_id = ""; //Your authgear app id
    const key_id = ""; //you authgear key ID
    const expiresAt = Math.floor(Date.now() / 1000) + (60 * 60); //the current value means token will expire in 1 hour.
    
    //Payload to include in JWT
    const claims = {
        aud: project_id,
        iat: Math.floor(Date.now() / 1000) - 30,
        exp: expiresAt
    }
    const privateKey = fs.readFileSync("key.pem"); //Read value from the downloaded key file
    const header = { "typ": "JWT", "kid": key_id, "alg": "RS256" }
    const jwt = node_jwt.sign(claims, privateKey, { header: header });

    return jwt;
}

See Admin API Authentication for a more detailed guide on how to get your key ID, and private key and generate Admin API JWT using different programming languages.

Step 2: Initiate User Export Task

Make an HTTP(S) POST request to the initiate export endpoint. To do that, first, install the node-fetch package in your app using this command:

npm install node-fetch

Then add the following code to your application:

const express = require("express");
const node_jwt = require("jsonwebtoken");
const fs = require("fs");
const fetch = require("node-fetch");
const app = express();
const port = 3002;

app.get("/export", (request, response) => {
  const jwt = generateJWT();
  const input = {
    format: "csv",
    csv: {
      fields: [
        {
          pointer: "/sub",
          field_name: "user_id",
        },
        {
          pointer: "/email",
        },
      ],
    },
  };

  const options = {
    method: "POST",
    headers: {
      "Content-type": "application/json",
      Authorization: "Bearer " + jwt,
    },
    body: JSON.stringify(input),
  };

  const appUrl = "https://your-project.authgearapps.com";

  fetch(`${appUrl}/_api/admin/users/export`, options)
    .then((result) => result.json())
    .then((result) => response.send(result))
    .catch((error) => response.send(error));
});

app.listen(port, () => {
  console.log("server started! PORT: " + port);
});

Step 3: Check the Status of a User Export Task

In this step, we'll check the status of the export task we initiated by making an HTTP(S) request to the Check Status endpoint.

Add the following route to your app to check the status of an export task using the task ID:

app.get("/status/:id", (request, response) => {
  const jwt = generateJWT();

  const options = {
    method: "GET",
    headers: {
      "Content-type": "application/json",
      Authorization: "Bearer " + jwt,
    },
  };

  const appUrl = "https://your-project.authgearapps.com";

  fetch(`${appUrl}/_api/admin/users/export/${request.params.id}`, options)
    .then((result) => result.json())
    .then((result) => response.send(result))
    .catch((error) => response.send(error));

  
});

When the status task is completed, the HTTP(S) response body will contain a download_url field. Open the URL in the value of download_url to download the exported users file.