Applied AI LLM Gateway

The LLM Gateway was developed to enable quick and easy access to LLMs for the purposes of experimentation and proof-of-concepts.  The goal of the platform is to enable teams to determine if an LLM is right for their use case without needing to manage or deploy their own models.  If a team wants to move forward with a use case, they will then need to go through the proper legal reviews and create their own cloud account to host the LLM they’re using.

More information about the use of Generative AI and LLMs at Comcast can be found at:

• Use of Generative AI at Comcast
• Comcast Principles for AI/ML
• Generative AI in Cloud

Guidelines for using the LLM Gateway

1. The gateway is only meant for experimental use cases and proof-of-concepts.  It should not be used or integrated with any production services.
2. To comply with Comcast legal standards, access is limited to 90 days per user.  This should be enough time for users to determine if an LLM is right for their use case.  After 90 days, the user’s access will be revoked.  Users can request access again, but continued access requests will be scrutinized to ensure there’s no misuse of the gateway.
3. You should not submit any personal information (PI) or sensitive data to the LLMs.
4. Be mindful of the number of requests you’re sending to the gateway.  Heavy usage increases operating costs and we may need to revoke access to anyone incurring a large spend.

Supported Models and EndPoints