CombineXPO.exe and importing the combined XPO

(0) Share

Report

Posted on by Richard K

I don't have expectations of any direct answers, but I'm really hoping that someone can help give me some insight or direction of where to go next. We're using Dynamics AX 2012 R2 CU7.

Part 1: The mystery of the out of order class import

As part of our automated build process, we use the combinexpo.exe utility that comes with Dynamics AX 2012 R2. When it does the work of combining all of our XPO's, it seems to do something a bit funky with a base class and it's derived classes. It puts the derived classes first and then puts the base class after during the import. Because of this, we end up with some errors during the import as the classes are being imported in the wrong order ( in my opinion ).

Here's what I see in the import log.

<Info>Imported AOT element: \Classes\PdsFormCtrl_InventTransferUpdateRemain</Info>
<Info>Imported AOT element: \Classes\PdsFormCtrl_PurchUpdateRemain</Info>
<Info>Imported AOT element: \Classes\PdsFormCtrl_SalesUpdateRemain</Info>
<Info>Imported AOT element: \Classes\PdsFormCtrl_UpdateRemain</Info>

I'm thinking that the PdsFormCtrl_UpdateRemain should be imported first.

Has anyone else seen this?

As well, I see this exception in the Event Log, but this is a brand new file that the CombineXPOs creates.

CombineXPOs.exe : Error [c:\somelocation\combinedmodel.xpo] System.IO.IOException: The process cannot access the file 'c:\somelocation\combinedmodel.XPO' because it is being used by another process.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at Microsoft.Dynamics.AX.BuildTools.CombineXPOs.XpoReader..ctor(String file)
   at Microsoft.Dynamics.AX.BuildTools.CombineXPOs.MainClass.addFileNoCatch(StreamWriter destinationFile, String filename, Boolean allowDuplicate)
   at Microsoft.Dynamics.AX.BuildTools.CombineXPOs.MainClass.addFile(StreamWriter destinationFile, String filename, Boolean allowDuplicate)

Part 2: The mystery of why the hell does it take so long to import the combined XPO?

Further along our automated build process, we do the work to import the combined XPO. It's imported via an XML startup file that used the XpoImport command and sets up logging.

<?xml version="1.0" encoding="utf-8"?>
<AxaptaAutoRun version="4.0" logFile="importmodel.log" exitWhenDone="true">
<XpoImport file="model.xpo" />
</AxaptaAutoRun>

The arguments to do the import look like this:

$arguments = '-aol={0} -aolcode="{1}" -lazyclassloading -lazytableloading -nocompileonimport -internal=noModalBoxes -model=@{2} -startupcmd=autorun_{3}' -f $layer, $layerCode, $model, $importXml

So, this process has typically been about 30 minutes, but after some minor code changes that were checked into TFS, this jumped to ~63 minutes. The changes were to existing items that were in TFS and were quite minor. I've scoured through the import log ( that's where I found the errors in part 1 ), but I cannot find any indicator of why this is happening. Nothing else changed on the server as far as I know and the build process has been zinging along without any changes for months and months. I've also been through the event log and the import log. I haven't seen anything that's an indicator. I've also manually imported the combined XPO and haven't seen anything different than the automated import.

All of the other tasks excluding the AXBuild.exe process take approximately the same time to complete ( DB Sync, CIL, etc ).

Is there anywhere else I can look to see errors? Could there be caching, settings, etc that's in Dynamics for the build user? Any ideas of where I can look, what I can flush, or what I can beat up would be appreciated.

I've been assured by our infrastructure group that nothing on our servers have changed that would cause this ( I know, I know, never trust server guys ).

*This post is locked for comments

I have the same question (0)

All responses (14)

Answers (1)

Brandon Wiese 17,788 on at

Like (0)

Report

Do you use an AOS cluster?

Was this reply helpful? Yes No
Richard K 50 on at

Like (0)

Report

Not on our build AOS. It's a physical AOS server with a virtual database server.

Was this reply helpful? Yes No
Suggested answer

Joris dG 17,775 on at

Like (0)

Report

I'm currently investigating a somewhat similar issue at an R3 customer. On the flip side, with the many builds I've done over the years, I've never seen this as an issue - but that could be sheer luck (or naming our class hierarchies alphabetically). However, I have some speculative things that would be great if you could try them.

The command-line import has a feature where it does multiple passes of imports. If an object is imported and has issues (like, a reference missing!) - it will be imported again after all the objects are imported first. It does several passes of this until all issues are resolved (or some threshold is met). So that, combined with the combineXPO that has logic to (supposed to) order objects in the XPO based on dependencies - this whole import shouldn't have trouble with dependencies. In fact, we used to use our own tool to combine XPOs, which didn't order objects at all. Using the official combineXPO is essentially an optimization - to avoid having to do too many passes of the import. If things are ordered perfectly, you may only need to import the XPO once.

Now, I'm questioning whether the multiple passes and how it detects these issues depends on compile. Considering your command line has the -nocompileonimport flag set I'm wondering if that has any impact on this. Would you be able to try a build without the nocompile flag and see how that impacts things? I would expect it take a bit longer to import but perhaps the dependency checking depends on compiling...

Was this reply helpful? Yes No
Richard K 50 on at

Like (0)

Report

I'll setup a test case tomorrow with the -nocompileonimport being removed from the import. It's too late in the day for me to be tinkering with the build machine. Thanks for the response.

Was this reply helpful? Yes No
Verified answer

Joris dG 17,775 on at

Like (0)

Report

Another comment. You mention you have a physical AOS with a virtual SQL. During a build the performance gain/loss is communication between client/server/sql. If you put them all on one box, you're going to be amazed at the performance gain. At my old job we used out-of-warranty laptops as build boxes - bought more memory to up the mem to 16gb, and put an SSD in there. Install client, aos and SQL local. Best build boxes ever. I think for R2/R3 average build times (clean environment, X++ compile with axbuild, CIL, synchronize) were less than an hour (depending on code base size between 40 and 60 minutes).

Was this reply helpful? Yes No
Richard K 50 on at

Like (0)

Report

Well, removing the -nocompileonimport argument didn't make a difference. My import time was still ~60 minutes.

I'll look into moving the AOS and SQL onto one box. That should help as well.

Was this reply helpful? Yes No
Joris dG 17,775 on at

Like (0)

Report

I would expect that without -nocompileonimport the import would take longer. The question is whether it solved some of your dependency problems? (part1 :-))

Was this reply helpful? Yes No
Richard K 50 on at

Like (0)

Report
It did not =(

<Error>*** Error: 9, Variable inventDimID has not been declared. (\Classes\PdsFormCtrl_PurchUpdateRemain\initPost; Line 20, column 5)</Error> <Error>*** Error: 9, Variable inventDimID has not been declared. (\Classes\PdsFormCtrl_SalesUpdateRemain\initPost; Line 14, column 5)</Error> <Error>*** Error: 9, Variable inventDimID has not been declared. (\Classes\PdsFormCtrl_InventTransferUpdateRemain\initPost; Line 15, column 5)</Error>

Was this reply helpful? Yes No
Richard K 50 on at

Like (0)

Report

[quote user="Joris dG"]

Another comment. You mention you have a physical AOS with a virtual SQL. During a build the performance gain/loss is communication between client/server/sql. If you put them all on one box, you're going to be amazed at the performance gain. At my old job we used out-of-warranty laptops as build boxes - bought more memory to up the mem to 16gb, and put an SSD in there. Install client, aos and SQL local. Best build boxes ever. I think for R2/R3 average build times (clean environment, X++ compile with axbuild, CIL, synchronize) were less than an hour (depending on code base size between 40 and 60 minutes).

[/quote]

This was the answer for us it looks like. I moved the build database to the actual build machine and we're down to 110 minutes for the full build ( we have some extra stuff in the build like checking for temporary labels and some best practices things ).

I think with some more tinkering I can probably shave another 20-25 minutes from the build as well.

Thanks for the help and the great suggestions.

Was this reply helpful? Yes No
Kenny Saelen on at

Like (1)

Report

Allow me to add a remark concerning the XPO import. There is an issue we are seeing right now that is still under investigation.

We use the CombineXPOs utility (it already handles some of the dependencies) but still we need to import the XPO file twice to resolve every dependency. But as importing twice works well, this is fine for us.

Yet, we have noticed issues in environments where we deploy the models that came out of the build. All of the objects related to security were imported as part of the created XPO during the build. Afterwards, when we deploy these models into a test environment, all of the objects are still there. So far so good.

But the issue begins when users log in and have limited access to some of the functionality. And when we investigated the issue, we saw that although the artifacts are there, there is a problem with permission. If we go to certain security roles where we have added custom privileges and we hit the override permissions button, we see that the list of objects in the tree does not contain all of the securable objects for that role.

We have a workaround by importing the XPO again in the test environment (manually) and than the list of securable objects contains the custom ones. So there is clearly something going on when importing XPO's as part of our build.

So you check the same in your builds to make sure this is ok. Once we have a solution (it's also submitted as a support ticket) I will post it here.

Was this reply helpful? Yes No