Restricting Internet Traffic

Application level authentication

Anyscale provides the option to put Serve deployments behind an authentication proxy to restrict traffic from unauthenticated users. By default, all clusters are started with restricting public internet traffic and only accepting authenticated user requests.

Enable traffic from public internet

To enable all traffic to Serve deployments, set allow_public_internet_traffic=True when creating a cluster through the SDK, or specify it in the Ray client builder API (ray.init("anyscale://cluster_name", allow_public_internet_traffic=True, autosuspend=-1)). All users will now be able to access the Serve endpoints at https://serve-session-{cluster_id}.i.anyscaleuserdata.com.

Authenticate users

To restrict traffic from unauthenticated users, set allow_public_internet_traffic=False when creating a cluster through the SDK, or specify it in the Ray client builder API (ray.init("anyscale://cluster_name", allow_public_internet_traffic=False, autosuspend=-1)). This is the default value for all clusters, so not specifying a value will require authentication for Serve. An authentication token will now be required to access the Serve endpoints.

The Serve authentication token can be obtained from the Anyscale CLI token as follows:

CLUSTER_TOKEN = requests.get(“https://api.anyscale.com/v0/clusters/{cluster_id}", cookies={"cli_token": ANYSCALE_CLI_TOKEN}).json()["result"]["access_token"]

The Anyscale CLI token is the same token you use to authenticate for Anyscale API calls, it can be found in the "Credentials" tab in the UI.

A user can then get authenticated by passing the Serve authentication token in the cookies when calling the Serve endpoints.

requests.get("https://serve-session-{cluster_id}.i.anyscaleuserdata.com/my_serve_endpoint", cookies={"anyscale-token": CLUSTER_TOKEN})

The Serve authentication token is unique to each cluster and can be shared in the URL to authenticate users who don't have access to the CLI token.

https://serve-session-{cluster_id}.i.anyscaleuserdata.com/auth/?token={CLUSTER_TOKEN}

This will redirect to the Serve service and place the CLUSTER_TOKEN in the browser cookies to authenticate future requests. Note: This functionality requires anyscale>=0.4.20 to be installed on the cluster.

Editing privacy settings for existing clusters

Updating the allow_public_internet_traffic argument for a running cluster will currently require restarting the cluster and redeploying the existing Serve deployments. This can be done through the SDK or through the Ray client builder API if (update=True):

ray.init("anyscale://cluster_name", allow_public_internet_traffic=new_allow_public_internet_traffic_value, update=True, autosuspend=-1)