Supply chain | Supply Chain Management, Commerce

Unanswered

Data management package API - Logic Apps - Zip issue

(0) Share

Report

Posted on by Saso GLIGOR

Hi all,

I was working on an integration project with Data management package API and Logic Apps.
If you have been involved in this kind of work, you might have stumbled upon the following issue.

Let’s say the source of the data to be integrated in FinOps is an FTP folder and files are dropped there by an external system. The logical way to proceed would be to create an import project in FinOps, Logic App which connects to the FTP folder, reads data and sends it to the FinOps import project.
Just to be clear, this is using the Data management package API, and not the Recurring integrations API, which is outdated as far as I know.

The problem that you might encounter is having a simple CSV file in the source FTP folder, and you need to zip it along with the Manifest and PackageHeader files in order to have a full package, before sending it to the import project in FinOps. Why this is a problem, because Logic Apps, very strangely, do not propose a zip action in the FTP connector. You can unzip files, but you cannot zip them in an FTP folder.

This is the way I got around this problem, and I wanted to share this solution to help, get you thoughts on this approach, and eventually a better solution J

I extended the DMFPackageImporter class importFromPackageV2 method in order to:

- Download the data file (the one which was read in the FTP folder).
- Generate the Manifest and PackageHeader files.
- Upload them to the BLOB.
- Give the new package URL to the method.

And it seems to work quite well. Tests were done, no trailing files left locally after execution and overhead of this supplementary process does not have a great impact on performance.

The execution of this extension is controlled by a keyword that I pass through the “ExecutionId” parameter of the Microsoft.Dynamics.DataEntities.ImportFromPackage service. This parameter is not mandatory and you can use it to pass miscellaneous information, like this one. The keyword, followed by the entity name is sent to the above service, and the rest is done in the X++ code.

Ex: Keyword “-MyKeyword-“, ExecutionId “-MyKeyword-MyEntityName”.

One other thing, it is working only for single entity import projects. For the moment, that is the requirement, but I should be able to customize it more and have multi entity import projects working as well.

You can find the extension here after.
Hope that helps someone.

Saso

Logic Apps HTTP request

2020_2D00_09_2D00_11-10_5F00_02_5F00_07_2D00_Clipboard.png

DMFPackageImporter extension

(For the code please see the posts below).

I have the same question (0)

All responses (5)

Answers (0)

Martin Dráb 237,880 Most Valuable Professional on at

Like (1)

Report

It looks like a blog post; not like something to be discussed in a discussion forum.

You can simply create a new blog right here on dynamics.com - see Request a New Blog or Blog to Syndicate for details.

Martin Dráb

237,880 Most Valuable Professional on at

Like (0)

Report

Trying to resend code in a readable form:

[ExtensionOf(ClassStr(DMFPackageImporter))]
final class DMFPackageImporterCALClass_Extension
{
    public DMFExecutionId importFromPackageV2(SharedServiceUnitURL packageUrl,
                                              DMFDefinitionGroupName sourceDefinitionGroupId,
                                              DMFExecutionId sourceExecutionId,
                                              boolean execute,
                                              boolean overwrite,
                                              DataAreaId legalEntityId,
                                              BatchGroupId importBatchGroupId,
                                              boolean failOnError,
                                              SharedServiceUnitFileID packageFileId,
                                              boolean _processIntegrationBridgeData,
                                              boolean _export,
                                              boolean internalCall,
                                              boolean _runAsyncWithoutBatch,
                                              int _thresholdToRunInBatch,
                                              int _previewCount,
                                              str _messageId)
    {
        DMFParameters dmfParameters = DMFParameters::find();
        
        if (dmfParameters.CALCustomImportFromPackageActive == NoYes::Yes)
        {
            str keyword = dmfParameters.CALCustomImportFromPackageKeyword;

            if (sourceExecutionId && 
                strStartsWith(sourceExecutionId, keyword))
            {
                str dataFileName = subStr(sourceExecutionId, strLen(keyword)   1, strLen(sourceExecutionId));
                DMFDefinitionGroup dmfDefinitionGroup = DMFDefinitionGroup::find(sourceDefinitionGroupId);

                if (dataFileName && dmfDefinitionGroup)
                {
                    // inspiration source : DMFPackageExporter.downloadDataProjectToPackage
                    Map exportedFiles = new Map(Types::String, Types::String);
                    exportedFiles.insert(dataFileName, packageUrl);

                    // Create local package file
                    str packageZipFileLocalPath = DMFPackageExporter::createExportPackage(dmfDefinitionGroup.DefinitionGroupName,
                                                                                          dmfDefinitionGroup.DefinitionGroupName,
                                                                                          exportedFiles);

                    // Upload package
                    str zipFileName = System.IO.Path::GetFileName(packageZipFileLocalPath);
                    SharedServiceUnitFileID newPackageFileId;
                    SharedServiceUnitURL newPackageUrl;

                    if (packageZipFileLocalPath && zipFileName)
                    {
                        newPackageFileId = DMFDataPopulation::uploadFileToBlobStorageAndGetUrlV2(zipFileName, packageZipFileLocalPath, '');
                        newPackageUrl = DMFPackageUtil::getDownloadURLFromFileId(newPackageFileId, 60, FileUploadTemporaryStorageStrategy::AzureStorageCategory);

                        // Delete the local file.
                        DMFStagingWriter::deleteFileOnServer(packageZipFileLocalPath);
                    }

                    packageUrl = newPackageUrl;
                    sourceExecutionId = "";
                }
            }
        }

        return next importFromPackageV2(packageUrl,
                                        sourceDefinitionGroupId,
                                        sourceExecutionId,
                                        execute,
                                        overwrite,
                                        legalEntityId,
                                        importBatchGroupId,
                                        failOnError,
                                        packageFileId,
                                        _processIntegrationBridgeData,
                                        _export,
                                        internalCall,
                                        _runAsyncWithoutBatch,
                                        _thresholdToRunInBatch,
                                        _previewCount,
                                        _messageId);
    }

}

Saso GLIGOR 70 on at

Like (0)

Report

Hi Martin,

Thanks for the suggestion, I will transfer this content to the blog post section.
I cannot get the “Inset code” menu to work, the posted content is unreadable.
Any ideas on what am I doing wrong?

Thanks,
Saso
Martin Dráb 237,880 Most Valuable Professional on at

Like (0)

Report

When I tried to edit your post, I got the same result as you. When I copied the same code to my reply, it works.

Sorry, I don't currently have time to explore it further. You can try it to discuss it in Community Feedback forum.
Community Member on at

Like (0)

Report

Hi Saso GLIGOR,

I do have same requirement and facing same issue. When I pass package URL then my File ID is not created i.e. 000000-0000-0000-00000000000 is getting created.

Can you help me the steps that you have used in logic app creation ?

Thanks in advance.

Join the conversation

Community site session details

Data management package API - Logic Apps - Zip issue

Helpful resources

Quick Links

Responsible AI policies

Neeraj Kumar – Community Spotlight

Congratulations to the November Top 10 Community Leaders!

Leaderboard > Supply chain | Supply Chain Management, Commerce

Featured topics

Product updates

Community site session details

Data management package API - Logic Apps - Zip issue

Helpful resources

Quick Links

Responsible AI policies

Neeraj Kumar – Community Spotlight

Congratulations to the November Top 10 Community Leaders!

Subscribe to this forum!

Select categories

Leaderboard > Supply chain | Supply Chain Management, Commerce

Featured topics

Product updates