I just got off the phone with support (for an hour) and there's been a HUGE change that was not communicated to developers.
THE APIs NOW ACCEPT DUPLICATE EMAIL ADDRESSES.
When your account moves to the new contact format (I'm not sure what percentage of customers are already converted), the APIs will no longer filter out duplicates. If you make a request to add a contact whose address already exists, it will be added even if the email address is already in your contacts.
This also affects bulk imports via the APIs! Beware!
Needless to say, I'm very upset that this wasn't properly communicated to developers. This will take me weeks to fix in my applications.
Thanks for the post on this. I want to clear up a few things regarding the API, bulk imports and contact duplicates. First off, the API does not currently support multiple contacts with the same email address. This is true for both v1 and v2 API and true for both single contact and bulk contact actions. I am truly sorry that you got incorrect information from our phone support team. They are not trained to handled detailed API questions and, as such, can often times give confusing or misleading answers. In the future, if you have questions about how the API does or doesn't work, the forums here or our email support (email@example.com) is the best option for getting API specific answers. As you can imagine, the API is far more technical than our normal product and our phone support staff just can't all be trained to answer those infrequent and very specific questions.
If you create a bulk import and have multiple entries with the same email address, the import will be processed in the exact same way. The duplicates will be detected based on email address and only email address. The last entry in the import with the same email address will "win", in that it will be the last update on the Contact and will be the remaining information that the Contact has. For example, if you imported firstname.lastname@example.org twice, first with the first name Bob, second with the first name Jane, the contact record with email@example.com will end up with the first name of Jane.
We have done extensive testing around this to ensure that there are no issues with this and that you can't have duplicate contacts created through the API as this would create an inconsistent and unpredictable experience from one customer to the next. This morning, we ran our tests in production again after this post to ensure that is still the case and we came back without any duplicates created. I just manually did this in my account and confirmed it as well.
Regarding the new Contact Management system you referenced, we are in fact migrating our customer base onto a new and far more advanced Contact Management system. This has been in development for a very long time and we've made many public mentions on earnings calls, press releases and communications to customers about this. One area where we haven't done a great job of communicating yet, and this was intentional, is to our developer ecosystem. This is because the majority of our developer ecosystem hasn't yet been impacted by this and we weren't ready to message them until they were going to be impacted. The communications are planned to cover what we're doing, how these changes impact integrations and how we can support our developer ecosystem and these communications will be going out in the not too distant future.
It is true, one of the new features that will be rolling out will be email address no longer being a unique field for Contacts. However, at this time we don't have this feature rolled out to 100% of our customer base and to prevent inconsistencies in the API we are not allowing this through the API. We will never be allowing this through our v1 API, it simply is too difficult to update that version to support this. However, we do plan to add this feature to the v2 API sometime in 2014 when we have all of our customer base migrated to the new CRM features. We will also be offering multiple email addresses per contact, multiple physical addresses, multiple notes and tagging. All of these features will not be available to the API until 2014 and the API, in the interim, will continue to behave exactly the same as it has over the past 6 years.
If you have any more questions about this new platform, feedback on it or concerns, please feel free to reach out to our API support team or PM me your contact information. I'm more than happy to have a call with developers to understand needs, concerns and discuss what we're doing. We always are looking for feedback from our customers on how to make the product better and my customers are you, the developer ecosystem. Again, I apologize for the inaccurate information you received on the phone. I will work with our support training team to get some more educational materials out to them and help prevent this from happening again.
I'm PMing you now. Your APIs have accepted duplicates for over a week. The proof was in my account this morning.
To be clear, adding contacts via the bulkUrlEncoded method from your V1 XML API .NET library allows duplicates. Here's a VB.NET example (directly converted from your C# library) that inserts duplicates (as of this morning at 7am CST):
''' <summary> ''' Uploads text/csv file dumped to string to specified lists ''' </summary> ''' <param name="authdata">Authentication Data</param> ''' <param name="data">CSV or text, dumped to string</param> ''' <param name="listIds">ID(s) of target lists to upload to</param> ''' <returns>Calls urlEncodedPost, which then returns response from server (string)</returns> Public Shared Function bulkUrlEncoded(ByVal authdata As ConstantContactUtility.AuthenticationData, ByVal data As String, ByVal listIds As IList(Of ConstantContactBO.QRLendingLists)) As String If String.IsNullOrEmpty(data) Then Throw New ArgumentException("No data to be uploaded specified.", "data") End If If listIds.Count = 0 Then Throw New ArgumentException("No target list ID(s) Specified.", "listIds") End If Dim encodeddata As String = "&data=" & HttpUtility.UrlEncode(data) 'SV_ADD is add contact, REMOVE_CONTACTS_FROM_LISTS self explanatory. CLEAR_CONTACTS_FROM_LISTS 'sent with no data will clear the list. Dim i As Integer = 0 Dim JoinedURIs As String = "" For i = 0 To listIds.Count - 1 JoinedURIs = JoinedURIs & "&lists=" & Convert.ToString(authdata.AccountContactListsUri) & "/" & listIds(i) Next Dim fullrequest As String = "activityType=SV_ADD" & encodeddata & JoinedURIs Try Return Utility.urlEncodedPost(authdata, authdata.accountActivitiesUri, fullrequest) Catch e As Exception Throw New ConstantException(e.Message, e) End Try End Function
''' <summary> ''' HTTP POST using URLEncoded Content Type ''' </summary> ''' <param name="Authdata">Authentication Data</param> ''' <param name="URI">URI to POST URLEncoded data to</param> ''' <param name="content">URLencoded data to POST</param> ''' <returns>string containing XML response from server</returns> Public Shared Function urlEncodedPost(ByVal Authdata As ConstantContactUtility.AuthenticationData, ByVal URI As String, ByVal content As String) As String ' Create a request Dim request As WebRequest = WebRequest.Create(URI) ' Set API+UN/PWD Credentials request.Credentials = New NetworkCredential(Authdata.AccountUserName, Authdata.Password) ' Set Method type request.Method = "POST" ' Create POST data and convert it to a byte array. Dim postData As String = content Dim byteArray As Byte() = Encoding.UTF8.GetBytes(postData) ' Set the ContentType of Request request.ContentType = "application/x-www-form-urlencoded" ' Set the ContentLength request.ContentLength = byteArray.Length ' Get the request stream. Dim dataStream As Stream = request.GetRequestStream() ' Write the data to the request stream. dataStream.Write(byteArray, 0, byteArray.Length) ' Close the Stream object. dataStream.Close() ' Get the response. Dim response As WebResponse = request.GetResponse() ' Get the stream containing content returned by the server. dataStream = response.GetResponseStream() ' Open the stream using a StreamReader for easy access. Dim reader As New StreamReader(dataStream) ' Read the content. Dim responseFromServer As String = reader.ReadToEnd() ' Clean up the streams. reader.Close() dataStream.Close() response.Close() Return (responseFromServer) End Function
If I include email addresses within the CSV [data] string that are already in my contacts, the email addresses will be added again.
Thanks for posting the code samples. Definitely looks like there is a problem with your imports not getting appropriate duplicate detection and we need to take a look at this. If possible, could you log the actual request being created by the wrapper library and sent in the HttpWebRequest to us and then email it to firstname.lastname@example.org? While we are definitely able to see something is off in your account, we still haven't been able to reproduce which will slow down our troubleshooting and getting a fix. Your payload would help us out greatly to eliminate as many variables as possible.
I send the content of the requests I'm sending to your webservices account. Let me know if you need anything else to help troubleshoot.
Thanks for sending the additional information. We have reproduced the error you described using our v1 bulk API. We are looking into why this is happening and why it's only happening for the v1 bulk API but not the v2 bulk API. Will hopefully have an update soon, however this is a holiday week with Thanksgiving so we are not sure if we'll be able to get this solved this week.
A workaround to avoid this issue is to use the v2 API instead of the v1 API. While we have deprecated the v1 API and it may have degraded experiences in the future as we continue to roll out Contact Management enhancements, this was unintentional and we do intend to fix this. In the long term, we do plan on retiring the v1 API and moving to the v2 API is highly recommended.
It does, of course, make sense to move to the V2 API if possible. Unfortunately, until you fix, or at least document the validation and filtering items for, the email_content error in the V2 API, there's just no chance that I'll be able to do that.
I'll consider moving some of my API calls to the new API, but it's impossible for me to use the new API for creating or updating campaign content until your validation is usable.
We have an update on the duplicate emails issue. In the investigation, it looks like we have a bug in one of our frameworks where if the only field provided in the bulk import is email address, or the last field provided is email address, than duplicates will be created. We are working on fixing this bug and also investigating how to correct the duplicate contacts in accounts with this scenario without any interaction from impacted customers. We expect to have a fix for this issue tested and in production in early Dec (currently targeting Dec. 10th).
The good news is that there is a simple workaround you can put in place today that will stop the problem from continuing. If you add an additional field, such as first name or custom field 15, to the request and simply leave the information blank, no duplicates will be created.
We have also identified all customers affected by this issue so we can proactively fix their accounts and ensure that no billing issues were created for any of these customers. If any billing tiers were changed because of duplicate Contacts, we will make sure to have our billing group apply credits to the accounts to account for the issue on our end. Due to the specific scope of this issue we have only identified a handful of affected accounts so far and are really greatful to you for pointing this out and helping us troubleshoot so effectively and quickly.
Regarding the email_content issues, we are actively working on adjusting the validation now to be much more open. We hugely appologize for the issues that is causing developers. The inconsistency on the validation in our UI and our API is unforgivable and we are working very hard to adjust it and get them in sync.