It’s been a while when I wrote my first Keboola component (in Czech). I improved it a bit during the time (like supporting incremental extracts), but didn’t have time for a big changes (like BULK API 2.0). Just lately got another request for change – the increments work nicely for „normal“ objects but not for historical. Reason being, that historical objects don’t have LastModifiedDate field which I brutally add to the WHERE clause.
Ok, that would be nice change, to allow for incremental upload to Keboola and still support custom WHERE clause. Would be great to have a dynamic UI of the component, so it shows only at the right time and so on. So I asked, whether we can make a dynamic UI. And after a bit of back and forth I found out, that I don’t have to update my code and can just use what Keboola offers.
3rd party components
Keboola has a library of components. Some of them are „real“ components, which connects to other systems, while others just work in combination – like zip files, add row numbers and so on.
I have no idea how to use them, but Najlos has, so he updated my normal configuration, to support incremental load any time. So now, my extractor just extract data which I specify (and I can use something like LAST_N_DAYS:5 to load just part of the data) and this processor will do the incremental load to Keboola.
{„parameters“: {„sinceLast“: false,„objects“: [{„name“: „Contact“,„soql“: „SELECT FirstName,RedNoses__Last_Donation_Amount__c FROM Contact WHERE RedNoses__MaxDonation__c > 10″ }]},„processors“: {„after“: [{„definition“: {„component“: „keboola.processor-create-manifest“ },„parameters“: {„incremental“: true,„primary_key“: [„Id“]}}]}}