vault backup: 2024-08-10 08:51:45

Work/2024-08-02 Order-level discounting email response to Dan.md

@@ -0,0 +1,41 @@
+#email #rough-draft #logos 
+
+Hi all,
+
+I have an update about this effort.  
+
+The royalty calculator has been updated to produce additional information that takes order-level discounting into consideration.  These calculations are _not yet_ reducing royalties figures.  Our first step was to get the calculator spitting out the right numbers so that we could do some retrospective analysis on the potential savings we might find.  This will help us validate the original hope of saving $2million/year and the value of applying these calculations to royalty amounts.
+
+I've shared a sheet link below, prepared by Justin Strandburg.  A couple of notes about this sheet:
+
+1. The data in the sheet _only_ includes sales from orders that had order-level discounts
+2. The past three months were queried.  So, this is data for May, June and July 2024.
+3. Sheet rows are grouped by rightsholder, month and then royalty formula.  Because of this, you will see multiple rows for the same rightsholder.
+4. Column H shows the potential royalties savings.  Summing Column H gives us the total, potential royalties savings for the past three months.
+5. Column H _does not_ take any "can be discounted" configuration into consideration.  That is, these potential savings assume all rightsholders allow us to discount royalties for all resources.  We know this is not true. 
+
+Summing column H gives us $71,342.11.  Extrapolating this number over a year gives us (very) roughly $300,000.  Considering that the past three months do not include increased sales from a launch year, we could generously double this amount to $600,000.  This brings us _extremely generously_ to only a quarter of the original hope of $2m/year.  We're assuming that the real savings will be considerably less than $600k/year.
+[https://docs.google.com/spreadsheets/d/1d43TneMSwzg6ZzBUZzvJyiUG4kWXT009tIbM4KLE5MU/edit?usp=sharing](https://docs.google.com/spreadsheets/d/1d43TneMSwzg6ZzBUZzvJyiUG4kWXT009tIbM4KLE5MU/edit?usp=sharing)
+
+Even if the savings-per-quarter are half of what the sheet shows (so, roughly $150k/year), I think this work is worth completing, especially as we move forward with subscriptions and look to implement "5% off for subscribers"-type discounting. However, we have one remaining hurdle: an accurate source of data for which royalties we are contractually allowed to discount.
+
+The real source of truth is the large set of rightsholder contracts in LinkSquares.  Unfortunately, even with LinkSquares tooling, it's hard to efficiently plumb these contracts for royalty discounting clauses. The contract wording is not standardized and is not always a simple "yes" or "no".  
+
+The Titles & Terms spreadsheet is a mostly-reliable proxy for our contracts.  However, it suffers from some accuracy concerns.  Primarily, a) it is manually maintained, b) in analyzing it, we have found contradictory "discountable" answers for the same resource, and c) as Jessica said below, the sheet's purpose is to inform base package building and, thus, the "discountable" values don't always mean "contractually able to discount royalties".  
+
+We have two ways forward that I see:
+1. Rely on Titles & Terms to inform the royalty calculator about which royalties can be discounted
+2. Implement a "positive confirmation" approach, see below
+
+In my opinion, basing our calculations on the T&T sheet comes with risks we don't need to take. The risk being that we end up violating our contractual obligations to our rightsholders while having no process in place to catch these errors except our rightsholders informing us so that we can fix erroneous payments. 
+
+I believe the majority of potential savings can be achieved with no uncertainty by using a "positive confirmation" approach.  The sheet I linked-to above indicates that there are only a handful of rightsholders for which we could potentially save a meaningful amount of money.  For those rightsholders, I recommend that we manually review our contracts to confirm which of their resources can and cannot have royalties discounted. The process might look something like this:
+1. Clear all "can be discounted" checkboxes in BusinessDesk for all active contract terms. This checkbox is, currently, purely informational and has no function in the royalty calculator. We can automate this.
+2. Update the royalty calculator to honor the "can be discounted" checkbox for royalty discounting purposes.
+3. Determine the top-savings-potential rightsholders from the sheet Justin produced and review the contracts we have for them.
+4. Manually check the corresponding "can be discounted" checkboxes in BusinessDesk to accurately reflect contracts in LinkSquares.
+5. Continue calculating potential savings in the royalty calculator and reviewing the output for non-yet-positively-confirmed rightsholders to trigger the positive confirmation process.
+
+I apologize for the lengthy email.  I hope it's helpful for you all to guide us on the path forward!
+
+Ben

Work/2024-08-05 Duplicate billing profile bug hunting (BEN-1564).canvas

@@ -0,0 +1,13 @@
+{
+	"nodes":[
+		{"id":"2ae4e170393087ae","x":-420,"y":-300,"width":620,"height":140,"type":"group","label":"Transformer determines to create BP"},
+		{"id":"441005acc47d9c36","type":"text","text":"SalesforcePersonAccountLoader","x":-200,"y":-60,"width":297,"height":50},
+		{"id":"bda4d7cee8a1fc1a","type":"text","text":"No Logos BPs found","x":-380,"y":-260,"width":250,"height":60},
+		{"id":"2a546673faf1e1ab","type":"text","text":"No Logos BPs match","x":-100,"y":-260,"width":250,"height":60},
+		{"id":"34d9d49694429d5f","type":"text","text":"CRM API creates Logos BP","x":-51,"y":100,"width":260,"height":50}
+	],
+	"edges":[
+		{"id":"201bc0e3711a8b39","fromNode":"441005acc47d9c36","fromSide":"bottom","toNode":"34d9d49694429d5f","toSide":"top"},
+		{"id":"85cf554274bc5fdf","fromNode":"2ae4e170393087ae","fromSide":"bottom","toNode":"441005acc47d9c36","toSide":"top"}
+	]
+}

Work/AcademicDesk integration to Salesforce.md

@@ -0,0 +1,22 @@
+#academic-desk #subscriptions #salesforce #bulk-licensing
+
+[[2024-08-08 (Thursday)]] 
+
+Luke and Josh are talking about AcademicDesk housing academic subscriptions data. This is news to me.  Is the Academic team planning on their own subscriptions engine?
+
+- [x] Verify that this is _not_ the case and that they are using vanilla subscriptions API engine stuff 
+      Randall confirmed that they're not doing _anything_ with subscriptions at the moment.  It's all bulk licenses.
+
+Luke showed me what he's working on Salesforce:
+- A group account will have a License info tab that shows:
+	- Subscriptions for the group with, supposedly, information that comes from AcademicDesk: number of seats, the type of subscription (various checkboxes), etc
+	- The claimed licenses: this is simply a list of SF person accounts tied to the SF group account that have a subscription where the Group ID on the subscription ist he same as the Faithlife Group ID of the SF group account
+
+Request from Justin Ullestad: https://docs.google.com/document/d/14WEYQO0QW7PuIu1re2p2kh5X0KJz_zv4o3e-NBQwRX8/edit?usp=sharing
+
+These questions are irrelevant because they're not using subscriptions for any kind of academic licensing.
+~~For this to work, it looks like we need:~~
+- [ ] ~~Are group subscriptions integrating to Salesforce?~~
+- [ ] ~~If so, is the group ID being populated on subscriptions?~~
+
+[[Enterprise subscriptions meeting]] - see notes there.  The academic team has been tasked with using subscriptions for their bulk license distribution--this would enable students and church members to upgrade and continue subscriptions beyond/outside their church or school providing it.  The minimize work and changes in the commerce engine, the current plan is to continue to use bulk licensing (not subscription-based) for academic license distribution.  But, we don't integrate this kind of thing into Salesforce.

Work/BigCommerce Notes.md

@@ -0,0 +1,7 @@
+#bigcommerce #logos 
+
+# Tax Provider API
+You can create tax provider apps that can integrate with a custom tax provider system. There are some reason (which I don't remember) that are preventing us from using the built-in Avalara tax app.  We use Avalara, but in some custom way that the app doesn't support.
+
+It looks like we already implemented this: [[PAY-2742] Update SalesTaxApi BigCommerce Tax Provider for new Pick List Strategy - Jira (atlassian.net)](https://faithlife.atlassian.net/browse/PAY-2742)
+

Work/Daily Log/2024-08-01 (Thursday).md

@@ -0,0 +1,11 @@
+#copilot #logos 
+
+Tried to use GitHub Copilot this morning for refactoring the CRM API ETL unit tests to my new testing methods, but it couldn't get it right.  Later in the day, it did great on a more focused refactor ask.
+
+Met with Hugo for my 1:1 and he went our the mid-year review with me.  He only had nice things to say about me, so that's encouraging.  One thing to work on that we discussed: removing some project management duties so I can focus more on technical work.
+
+Had my 1:1 with Ricardo today too.  We mostly discussed the replatforming work: how it's going, his perspective and whether I'd like to get more involved with it in phase 2.  Summary: disappointed with Celigo, BigCommerce and Trellis (their quality of work), but generally happy with other things like Builder.io and using Next.js for new.logos.com
+
+Refactoring these CRM API ETL unit tests took most of the day--it's just a lot of manual effort and then fixing unit tests when the expectations are slightly different because I've revamped the testing helper methods.  I expect to be done today though.
+
+Tomorrow, I hope to get tests written for the Commerce Info ETL refactor and get a PR in for that.  

Work/Daily Log/2024-08-02 (Friday).md

@@ -0,0 +1,7 @@
+#unit-testing #logos 
+
+Merged my unit testing refactor PR.
+
+Created tests for the Commerce Account Info integration ETL refactor and put up a PR for that.  Will hope to merge it Monday.
+
+Had a good first book club meeting about the Excellence Wins book. I'm liking this book a lot--very easy to read and great thoughts on excellence in business and especially customer service.  Seems everyone on the call was enjoying it too.

Work/Daily Log/2024-08-05 (Monday).md

@@ -0,0 +1,9 @@
+#royalties #logos 
+
+Enjoyed [this article](https://graphite.dev/blog/bors-google-tap-merge-queue?ref=dailydev) about the problem of merge skew and how to solve it for high-velocity code repositories.
+
+Started working on bug causing duplicate billing profiles to be created from Salesforce (CRM API) integration.  Not sure of the exact cause yet, but at least one user has 88 duplicate address-only BPs!
+
+Discussed the path forward for order-level discounts and determining which resources they apply to with Ezekiel and Kyle.  We agreed on not trusting the Titles&Terms spreadsheet, but giving the royalties team the tooling to accomplish this without making the decision as the dev team on which data source to trust.  This puts that responsibility where it belongs (with the royalties team) and not with us, where we are poorly suited to make that decision or take responsibility for the outcome.
+
+Nate Merritt told me he is leaving Logos and going to [BiblioNexus]([Open Source Software for Bible Translation (biblionexus.org)](https://biblionexus.org/)).  This is a big blow to Logos and the Business Solutions Group in general.  Ricardo wants me to be the new tech lead for Commerce.  I'm willing, but tentative about doing it.  I can't fill Nate's shoes, for sure.

Work/Daily Log/2024-08-06 (Tuesday).md

@@ -0,0 +1,9 @@
+#logos 
+
+Nate Merritt announced he's leaving Logos today. I talked to my team about potentially becoming the tech lead for Commerce and the need for leadership for the Business Enablement team. Kyle and Ezekiel both are willing.
+
+Our internet went out pretty early in the day.  I need to reschedule 1:1s with Kyle and Ezekiel tomorrow.
+
+I couldn't continue debugging the duplicate billing profile issue because I needed to continue to look at BusinessDesk accounts, Graylog and the database.  Couldn't do that without internet.
+
+Instead I got a great head start on refactoring the Notes integration between Salesforce and Faithlife.  I have the rough draft done of the outbound (Faithlife to Salesforce) integration.

Work/Daily Log/2024-08-07 (Wednesday).md

@@ -0,0 +1,9 @@
+#logos 
+
+Internet was out for most of the day, so I worked from the dining room table so that my phone had a good enough reception to do my hotspot.  It doesn't work well in the basement.  I'm guessing it's because the reception isn't as good.
+
+Met with Nate today about possibly taking over the commerce team tech lead position with him leaving for BiblioNexus. Sounds like I could do it.  It wouldn't be much different than leading the BEN team.  But, even though I'm not the one who would fill this role, the biggest hole he's leaving is the staff engineer position and his historical knowledge.
+
+Deployed the fix for the salesforce person account integration test fixture that was causing duplicate billing profiles sometimes.
+
+Deployed a fix for segments being pushed to Amplitude cohorts.  The bug was that we were accidentally replacing the entire cohort population when the cohort name changed but weren't pushing the entire population to Amplitude.  The fix was to push the whole population any time the name changes.  The Amplitude Behavioral Cohort API doesn't have a route to just update the cohort without replacing the population.  Once deployed, I reran the Salesforce Data Loader job to re-upload all the Amplitude cohort populations.  Going forward, it should keep them in sync properly.

Work/Daily Log/2024-08-08 (Thursday).md

@@ -0,0 +1,19 @@
+#logos 
+
+## [[Enterprise subscriptions meeting]]
+#joe-caun #nic-hoza #nate-merritt #ricardo-blanco #michael-fisher #randall-knutson
+
+Nic Hoza knows most of what's going on here. Basic desire for users:
+1. To see the subscriptions they have through their church or school
+	1. Current usage of bulk license distribution doesn't show anything to an individual user (they just get the license access)
+	2. Using bulk-license-delivering subscriptions still doesn't show the user this subscription since it belongs to the group
+2. Upgrade their church- or school-provided subscription with the difference of higher subscription
+
+## Commerce backlog grooming meeting
+#michael-fisher #ricardo-blanco #nate-merritt #kate-neale
+
+## Meeting with Kyle K.
+#bigcommerce #kyle-kennaw #replatform 
+It's very unlikely we'll move away from BigCommerce.  Currently painful, for sure, because of the dependence on our systems.  But, perhaps we can map out the future of the migration to get less and less dependent and entangled.
+
+I need to read more about BigCommerce and what it can do to get a better, informed picture of how we can move to it.

Work/Daily Log/2024-08-09 (Friday).md

@@ -0,0 +1,42 @@
+#logos
+
+# Quotes code refactoring in CRM API 
+#quotes #crmapi
+Refactored the AdminQuotesController in CrmApi today. It queries quotes from Logos (OrdersApi) and from Salesforce.  I like the pattern of having two interfaces: ILogosQuotesService and ICrmQuotesService to abstract both sources but admit that most of our code is aware that we're getting quotes from two places and so it makes sense for it to reference interfaces tied to one side and the other instead of a single interface the somehow abstracts both sides of the fence.
+
+After refactoring, the orders API client in my implementation of ILogosQuoteService is returning unauthorized. The problem was that I set it up in the CrmApi.v1 TypeRegistry as a Singleton instead of HybridHttpOrThreadLocalScoped.  That won't work because it has to pass credentials per request.
+
+# Deleted Opportunity problems in CRM API
+#quote #crmapi #salesforce #apex
+The CRM API integration tests were failing because the test quote couldn't be integrated to Salesforce UAT sandbox because a record with the same external id (our Logos quote ID) already existed, but was soft-deleted.
+
+I was having trouble finding the soft-deleted record.  [This article]([How to find deleted records in Salesforce using SOQL | Wipfli](https://www.wipfli.com/insights/articles/tc-how-to-find-deleted-records-in-salesforce-using-soql)) helped.  The answer is to execute developer console Apex code:
+```apex
+List<opportunity> deletedOpps = [SELECT Id
+FROM Opportunity
+WHERE LogosQuoteId__c = '6667952' and IsDeleted = TRUE
+ALL ROWS];
+
+system.debug(deletedOpps);
+```
+
+The `ALL ROWS` part is the key.  But, it doesn't work with a regular SOQL query, just from Apex code like this.
+
+I had to insert a new record manually because the upsert we were doing in the quote loader was always failing with an `ENTITY_DELETED` error.  I used Apex (the developer console in Salesforce) to 
+execute this:
+```apex
+Opportunity opp = new Opportunity(Amount = 10, Name='bens test', CloseDate = Date.newInstance(1960, 2, 17), StageName='Negotiating', LogosQuoteId__c = '6667952');
+insert opp;
+```
+
+Now the integration test passes.
+
+The key findings I had to research to get right:
+1. The list of parameters to the sobject (`Opportunity`) are comma-separated and formatted as `key = value`
+2. Create a new Date with `Date.newInstance(year, month, date)`
+3. Use `insert nn` to insert the object
+
+# Segment/Cohort Count Scheduled Check
+I implemented a scheduled check in CRM API to compare segment population counts from our DB to the user counts in Amplitude cohorts.  The idea was to highlight cohorts where the population difference was difference enough to indicate sync was failing somehow.
+
+Unfortunately, this approach didn't work.  Some cohorts were even over 50% off from the segment population count. I'm assuming this is because the users are just not present in Amplitude, which is very possible.  I removed the scheduled check for now until (if) we can find a better solution to check on these syncs.

Work/Enterprise subscriptions meeting.md

@@ -0,0 +1,5 @@
+#academic-desk #subscriptions #meeting #logos
+
+Content paid for by institutions
+Want students to be able to pay for bigger subscriptions
+

Work/Logos re-platforming.md

@@ -0,0 +1,31 @@
+#replatform #celigo #builderio #algolia #vercel #nextjs #bigcommerce
+
+Re-platforming efforts:
+1. Algolia to replace in-house elastic search implementation
+2. Builder.io to replace in-house built ContentAdmin CMS
+3. BigCommerce to replace various aspects of orders, checkout, cart, payments
+4. Vercel/next.js to replace Skeletor
+5. Stripe to augment payments, replace current payment providers
+6. Celigo for data moving, system integration
+
+## Algolia
+According to Nate Merritt, it's pretty much a disaster.  It's "working", but one of the major features we had to keep was the ownership search where users can filter out or in products they already own.  Algolia can't handle the thousands of products that users own despite what they said some solutions would be at the beginning of discovery.
+
+## BigCommerce
+I think this is a major disaster.  Almost none of the "just works" plugins work (search, pricing, taxes).  We're going to have to reach into our APIs to do dynamic pricing, discounting, taxes, etc.
+
+## Builder.io
+I'd say builder is a big success--or will be.  ContentAdmin is clunky and builder.io is smooth and has many tools to improve content management.
+
+## Vercel/next.js
+Next.js is a great framework and I think migrating to a more modern platform is a good idea.  Vercel has great deployment tools.
+
+However, I'm not sure it's leaps and bounds better than Skeletor--especially since it already does everything we want, we have great deployment support, etc.
+
+The BigCommerce catalyst site is not worth the trouble. The supposed gains of it already talking to BigCommerce are undermined by the fact that BigCommerce doesn't do everything we want out of the box and we have to talk to our custom APIs anyway.
+
+## Stripe
+A great platform, I think it's a good investment.
+
+## Celigo
+I'm not sold on Celigo at all.  Every time I've tried to do something with it, it doesn't work the way I need it to and I need a ton of custom code just to get it working.  I'd rather write a custom integration where I have Git support, PR review and visibility into what code is there instead of it being hidden in non-type-safe Javascript in a Celigo flow.

Work/Private Equity thoughts.md

@@ -0,0 +1,13 @@
+#privateequity #covehill
+
+My meeting with Nate yesterday triggered a few more thoughts on private equity and Cove Hill's ownership of Logos.
+
+1. Their goal is, obviously, profit
+	1. Moving to subscriptions shows a stronger revenue stream
+	2. While it both benefits us and our customers, subscriptions are driven mainly (I'm assuming) by Cove Hill wanting to show a strnger company to sell us to someone else
+2. There's no going back after selling to private equity.  
+	1. I don't know how Logos would ever ben independent again
+3. The re-platform to third-party tools is a mixed bag
+	1. Driven by resume building for Cove Hill, "our portfolio includes BigCommerce, Algolia, etc!"
+	2. We don't want to own tools that others make better.  But our business model is complex (ownership model, dynamic pricing, etc) and seeps into everything we do (orders, payment plans, subscriptions, etc) that third-parties do "better", but can't replicate for us
+	3. Moving to third parties is not going well