ADLS2 - Warning: Number of columns in source data file does not match the number of attributes

Like (0) Share

Report

Posted on 2 Jun 2022 11:56:33 by Johannes Wagner

Hi Community,

I am facing the following issue: I have a CDM folder in ADLS2 with a "clean" test data table (generated by Power BI dataflow, so the format should be ok).

I can import this datasource using PowerQuery and the ADLS2 connector (CDM) with no problems.

When I import the same CDM folder connecting directly to the ADLS2 (import method) "Azure data lake storage" it always completes with the following Warning:

"Number of columns in source data file does not match the number of attributes: [n] defined in the source data schema for the entity '[tablename]'.

Any thoughts on this are appreciated.

Thanks!

All responses (2)

Answers (0)

Johannes Wagner 45 on 02 Jun 2022 at 13:58:26

Like (0)

Report

RE: ADLS2 - Warning: Number of columns in source data file does not match the number of attributes

Thanks Bill!

I have checked the schema in all places (ADLS/model.json, CI, PowerQuery) and it is identical everywhere. It is a very simple test table so this can be easily verified.

And both ADLS and Powerquery read from exactly the same source.
Bill Ryan 8 on 02 Jun 2022 at 13:55:13

Like (0)

Report

RE: ADLS2 - Warning: Number of columns in source data file does not match the number of attributes

My first guess is that it's schema drift if it's CSV but if it's parquet than that's unlikely. If you look at the Column Count in the Dataflow, can you confirm it's the same number of attributes that are being defined? It's possilbe that the manifest.json/model.json file didn't get updated at some point, so you can always downloand and check it and modify if you see something else is in there.