Authentication for non-user-interactive integrations

[Rez]

I had a phone and email discussion with Eric Houston about this - while he assured me that I wasn't the only person who had asked about it, I didn't see anything explicitly relevant in the boards.

I develop automated/unattended/non-interactive integrations.  These integrations are usually triggered by webhooks in the source system.  Example:  a new customer record is created in an eCommerce platform; that event triggers a webhook, allowing the customer to be propagated to one or more other systems via secure automated integration brokering.  Please note - I am not looking for or asking Constant Contact to create webhooks or trigger events!

Today, virtually every vendor/provider supports this type of integration.  They do so by providing one or both of the following authentication protocols:

Basic Auth.  Constant Contact has dropped this capability citing security concerns; for user-interactive activities, you’re right, OAuth 2 is more secure.  For unattended integrations, Basic Auth continues to be the industry standard, and since API keys and secrets are passed encoded and within headers encrypted via HTTPS, security really centers around keeping API keys and secrets secure, which is and always has been (and always will be) the developer’s responsibility.  This protocol consists of encrypted credentials being passed for each API call.

OAuth 2, Grant Type Client Credentials.  This grant type was designed to authenticate access outside of user context, which fits the unattended model; however, this OAuth 2 grant type doesn’t appear to be enabled in the Constant Contact V3 API.  This protocol consists of encrypted credentials being passed, a token being returned, and the token then being passed for as many API calls as required at the time.

At this time, I have integrations in production to/from many popular/widely used systems, all of which support either Basic Auth, OAuth 2 Client Credentials Grant, or both; some examples are:

Microsoft (the entire Office 365 ecosystem)

Mail Chimp

ShipStation

Zendesk

Google (multiple API suites)

LightSpeed eCom

Shopify

SalesForce

As I mentioned to Eric, Constant Contact is literally the only platform I've been asked to integrate with that does not offer one or both of these protocols.  If Basic Auth is off-putting for whatever reason, then why not support OAuth 2 Client Credentials Grant?  It's part of the OAuth 2 spec along with the other flows, and is present in the OAuth 2.1 draft spec as well.

Eric had the impression that this might be addressed in the future, but I wanted to bring it up in this forum as well.

[SteveJ286]

Help me understand why their oAuth 2 "Server Flow" doesn't meet your requirements.  I'm starting on implementing it now and don't want to get half way there to discovered I've missed something.  Granted, I just need the contact list manipulation APIs.

[Rez]

Constant Contact's "Server Flow" doesn't support unattended non-interactive authentication.  It is in many ways similar to their "Client Flow", where multiple exchanges occur, including redirecting to a web page for credentials.  For web apps, this is completely normal; for true point-to-point (or m2m) integrations, there is no redirect location, and there is no user entering credentials.  Thus, neither "Server Flow" nor "Client Flow" will support such integrations without significant kludging and workarounds.

For context, here's a typical workflow for integrations I develop and support:

An Ecom system has a webhook to detect customer creation.  The webhook calls an API in AWS (the call is authenticated).  The API calls a Lambda function written in Node.js (just my preference).  The Lambda function:

Parses the incoming JSON payload and constructs JSON payloads for downstream systems (Constant Contact, Mail Chimp, etc.).

Pulls credentials from Systems Manager Parameter Store (as secure as it gets).

Calls each downstream API with appropriate payload, recording the result.

Logs the event, including sparse data about the customer and the results of the API calls.

Every data transaction is secured via https, and all credentials are stored securely.  At no time does a customer (or me for that matter) interact with the integration, aside from receiving a notification via text or email if they wish.

There are authentication methods specifically designed for this type of operation - the two I mentioned in my post.  For OAuth2, the contributors intentionally included the Client Credentials grant type to accommodate m2m authentication.

Hopefully that wasn't too long-winded.  Let me know if you have any other questions!

[SteveJ286]

Thanks for the reply.  I think for me, the current capabilities will suffice...that is, the client of my application does have to interact to give my app permission to access their account information(via the api).  After that setup, I can, via the refreshToken, use the API to interact with CC with no interaction from my client.

[EmmaH41]

I made a post about this use case almost a year and a half ago, they have done nothing. I don't come to this board very often but I always see posts about this.

[Rez]

I've concluded that Constant Contact has decided to not support it, for whatever reason.  I've lost two opportunities in the last year due to not being able to provide integration to this company.

[SaraGM]

I am trying to implement this exact thing and I've run into the same issue. Mine is an internal problem and this is not an issue I've encountered with other products (as you mention). This creates a significant bottleneck in integrating different systems and allow for automation of processes.  I would imagine this function is critical for many people.

[Rez]

The number of people and companies impacted by this is not insignificant, for sure.  Interestingly enough, Microsoft is just starting a project to deprecate Basic Auth for some of their Office 365 functionality - they have initiated an exhaustive communication campaign to ease the transition process for integrations.  They point directly to the Client Credentials OAuth2 grant type as their preferred method for m2m/unattended integrations, which lines right up with the stance of nearly all major cloud vendors.

[SCPEBA2012]

I'm running into exactly the same kind of situation. We have a (rather complicated) internal utility that we use to synchronize contacts on specific lists in our account. We've been using the V2 API to do this for years, but I realized recently that Constant Contact is making a push to get rid of the V2 API (you can no longer create new app keys for it). So, I started looking in to V3. First hurdle -- must go to OAuth2. 

Our utility is run unattended, after-hours. There is no true UI (though there is a very basic console-based UI for troubleshooting purposes). After digging into the options available, I'm left wondering how in the world to use the server flow without literally building a web application to receive the redirect -- all just to maintain the authorization. I considered hacks -- just make an invalid redirect URL, open a browser, and capture the HTTP traffic in web developer tools, then copy the keys around manually, maybe -- but this is not sustainable. It may get me working initially, but I have to make sure the process is easily documented so that other developers can handle the process if I'm not available.

At this point, I'm going to hold off on trying to adopt V3, and I may contact Constant Contact support directly to push for either limited bearer token (basic) auth or OAuth2 Grant Type Client Credentials (which I need to research). It may not amount to much, but another voice can't hurt.

[Courtney_E]

Hello SCPEBA2012,

Thank you for reaching out to Constant Contact API Developer Support. My team is here to assist outside software developers with questions about building into Constant Contact's API.

After you complete the OAuth2 server flow, Constant Contact returns an access token and refresh token. You can use access tokens to make requests using the V3 API. You can use refresh tokens to obtain new access tokens without any user input. This ensures that you only need to authenticate the account you’ll be connecting to once. After the initial authentication, you’ll just use the refresh token and access token to complete your requests.

V3 API OAuth2.0 Server Flow
https://v3.developer.constantcontact.com/api_guide/server_flow.html

Tokens Overview
https://developer.constantcontact.com/api_guide/auth_overview.html#tokens

[SCPEBA2012]

---

This ensures that you only need to authenticate the account you’ll be connecting to once. After the initial authentication, you’ll just use the refresh token and access token to complete your requests.

---

Yes and no. Certainly this is true as long as everything works correctly. But if the refresh token ever gets out of sync (e.g. I use it to generate an access token, but the response gets lost in transit due to a network problem), I will need to go through the initial steps again to get a new refresh token. An important part of all this is that the system must be maintainable by other developers in the case of my unavailability (the proverbial "if I get hit by a bus" scenario). That means I need to -- at minimum -- document the process of re-authorizing and generating a new refresh token. This leaves me with two options:

• Automate the re-authorization process (impossible without literally developing a web application to provide the web-based UI server flow expects)
• Document a manual process (possible, but ugly, since it would require steps like "generate the URL with a dummy redirect_url, turn on developer tools in the browser and capture the HTTP traffic).

Or am I missing something?

[Courtney_E]

Thanks for your reply! Your feedback and experience with this request is essential to improving our product, so thank you for reaching out to us regarding this matter.

We're currently making updates to our authentication system. This work would need to be completed prior to implementing additional authentication flows, however we are looking into them as future options. Some of the options currently under consideration include: Client Credentials flow, the Device Authorization flow, static refresh tokens, etc.

If you have a specific use case or auth type that you feel would better meet your needs, we’d love to hear from you. If possible, please include what specific solution(s) you’re looking for, whether your app would only access your own Constant Contact’s account data or if it would also be used by other accounts, etc. You can reply here, or contact us directly via email at webservices@constantcontact.com.

[SCPEBA2012]

This is great news! Thanks for the feedback on future plans.

Specifically for our situation, the main problem is that the existing OAuth flows are tied (at least during authorization) to an assumption that they are initiated from a web application of some sort (thus the requirement of a redirect URL). Since our utility is command-line only, a client cannot "redirect" to it. To be sure, I'm not an expert on OAuth 2.0, so I'm not entirely certain what flows would be best in our situation... some thoughts:

• The Device Authorization flow (as I understand it) is interesting to me as I can at least imagine how the authorization would work in a command-line situation (polling an endpoint seems... painful, but can certainly be accomplished).
• Client Credentials also seems interesting, though it may be dependent on implementation as the documentation I was looking at still included a redirect URL. I think, however, that you could provide any URL (maybe just "https://constantcontact.com"), as it doesn't necessarily return any information you would need when getting the access token. Constant Contact could also provide a simple page to use as the redirect URL in situations such as ours.
• I'm not sure what "static refresh tokens" means exactly, but if the idea is simply to alter the Server flow such that the refresh token doesn't change each time a new access token is requested, that would certainly also reduce the problem for us as it would be unlikely for a refresh token to become invalid (though I wonder if that is allowed under the OAuth specifications). In such a scenario, I would think a long (or configurable) expiration would be in order to periodically force new refresh tokens to be generated.

Probably more useful to you is our use cases:

• The ideal use case is that a static set of credential information can be stored in secure location (a file, database table, etc.) and is all that is necessary to connect to the API in a repeatable manner. If that connection requires exchanging the credentials for an access token (which certainly seems more secure), that's absolutely fine, but would not require the utility to receive a web request of any sort (e.g. the redirect we see in the Server flow). In this ideal scenario, other than setting up the App on the Constant Contact site (MyApps), no further interaction would be needed to authorize the utility.
• Acceptable approaches would soften the stance on user interaction during authorization. The utility could either be pre-authorized entirely on the Constant Contact site (and any necessary credentials or tokens manually recorded into its configuration), or it could initiate an authorization request that is entirely completed on the Constant Contact site (again, manual copying of credentials or tokens is fine).

It's worth noting that the current Server Flow is close to our needs. The biggest problem with it is the reliance on a web application (the target of the redirect URL) during the authorization flow. Since I don't have an appropriate internal target for this redirect, I'm left giving it a dummy URL and manually watching HTTP traffic in DEV tools, using a third party tool (e.g. Google's OAuth Playground), or developing a purpose-built web application just for the task. A very simple workaround (for us, at least) would be for Constant Contact to simply build functionality on their own site (somewhere in MyApps) that would allow me to move through the flow to the point of receiving a refresh token (basically taking the place of OAuth Playground).

We are moving forward with our work migrating to V3 using the server flow and OAuth Playground. I will, however, be interested to see new developments for alternative authentication methods as the become available (and will modify our implementation if any are appropriate for our uses).

Thanks for taking my feedback into account!

[Courtney_E]

Thanks so much for your feedback. It has been submitted for review and consideration by our engineering team. Your feedback and experience with this request is essential to improving our product, so thank you for reaching out to us regarding this matter.

[SCPEBA2012]

As an addendum to my response (primarily in hopes it will help others in my situation), I think I may have found an acceptable method to manually generate the refresh token when needed without building a custom web application. Google has an excellent tool for testing with OAuth2 APIs (OAuth 2.0 Playground). It's primarily geared towards Google's own APIs, but can be used with custom settings (click the Cog at the upper left and choose "Custom" under "OAuth endpoints"). You have to add https://developers.google.com/oauthplayground as a valid redirect url on your Application on Constant Contact's side for this to work. Put in the necessary information, go to the left and enter the desired scope(s), then click Authorize. As you step through the tool, eventually you get to where you have a refresh token and access token. At this point, the refresh token can be entered wherever it's needed (in our case, it will have to be recorded in a database table somewhere, since each use generates a new refresh token).

All that said, this makes my second option (document a manual process) reasonable. However, I do still think it would be a much-needed/desired enhancement for Constant Contact to implement the OAuth2 Client Credentials Grant flow. And I can easily imagine use cases where this would be a necessity (imagine a situation where someone not technically minded had to regenerate a refresh token!). 

I also dislike the dependence on a third party to help me get the refresh token. I suspect I can trust that Google won't be removing the OAuth playground tool any time soon, but if they do, I'll be back in the same boat I was before I found the solution. As an alternative, I suppose Constant Contact itself could provide an internal redirect URI page to facilitate generating refresh tokens when a reasonable redirect_uri can't be identified (or just create an interface on the app page that let you generate them as needed). Certainly this would be a quicker implementation than building out support for a new flow, though it's really more of a workaround than an actual solution.

[Courtney_E]

Thanks for your additional feedback!

[SaraK037]

Has there been any resolution to this issue? I'm trying to use Microsoft Flow to automate various Constant Contact API calls on a regular schedule so we can pull information into a database. Looking at the Swagger definition for the CC API, it appears that the OAuth2 Client Credentials flow is still not supported. 

My Microsoft Flow works for a handful of minutes/hours, but it requires me to manually refresh the connection every day by logging into my account via web browser using the "accessCode" flow. 

This obviously isn't sustainable, so any help is greatly appreciated. We really need a way to pull our data from the CC API automatically without human intervention! 

[Courtney_E]

Hello SaraK037,

Thank you for reaching out to Constant Contact API Developer Support. My team is here to assist outside software developers with questions about building into Constant Contact's API.

As Power Automate / Microsoft Flow is a 3rd party product and not built or supported by Constant Contact, we are limited in the support we can provide, but will try to answer any questions to the best of our ability as they pertain to Constant Contact’s API endpoints and functionality.

From what we understand, the issue users run into regarding Power Automate / Microsoft Flow not refreshing the token set for the V3 Oauth2 Server Flow is suspected to stem from their system expecting an access token in JWT format. An error seems to occur while attempting to parse the token, which leads to the refresh failure.

I was able to find this related community post in Microsoft’s public Power Users forum that may offer some additional insight on possible resolutions (link below), but if you need further assistance with setting up an Oauth2 authentication within Power Automate / Microsoft Flow in a way that supports refreshing the token set, you’d want to reach out to their support team directly for assistance: 

https://powerusers.microsoft.com/t5/Connecting-To-Data/How-to-connect-custom-API-using-own-OAuth2-in...

Additionally, we have been working on some updates for our authentication system. We're still finalizing these changes, and plan to communicate the details within the next couple of weeks. I suggest keeping an eye on our documentation for those upcoming announcements:

https://v3.developer.constantcontact.com/api_guide/release_notes.html

Please have a look and let us know if you have any other questions!

[SaraK037]

Hi Courtney, 

Thank you for the reply. I did happen to come across that Microsoft post a few weeks ago. Unfortunately, it looks like that user has direct access to the underlying API itself, so he/she was able to modify the API authentication method to work with Microsoft Flow. 

From what I'm seeing in the OAuth docs, it looks like the Client Credentials Flow needs to be specifically supported by the API in order for this type of unattended API call to work. 

https://auth0.com/docs/get-started/authentication-and-authorization-flow/which-oauth-2-0-flow-should...

[John__B]

Hello SaraK03,

Thank you for reaching out to Constant Contact API Developer Support.

Regarding the Power Automate / MS Flow, the upcoming authentication changes mentioned by @Courtney_E are going to include moving to the use of JWT access tokens, which should simplify the usage of this tool as it expects tokens in JWT format by default. 

While the pending changes will not include implementation of the Client Credential flow, we’re still investigating several options for future updates, to help accommodate non-interactive integrations while also maintaining user account security. 

Please let us know if you have any other questions!

Regards,

[NateP57]

Any updates from CC team on alternate flow support for unattended API integrations?