[ENH] Implementing the iTransformer model in PTFv2.

[JATAYU000]

Reference Issues/PRs

Fixes #1899

What does this implement/fix? Explain your changes.

Have started interfacing iTransformer in PTFv2, from the TSlib repository thuml/iTransformer
Work in progress, would like to have suggestions on it.

What should a reviewer concentrate their feedback on?

• Current Implementation compliance with PTFv2

Did you add any tests for the change?

Not yet

Any other comments?

PR checklist

• The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.
• Added/modified tests
• Used pre-commit hooks when committing to ensure that code is compliant with hooks. Install hooks with pre-commit install.
  To run hooks independent of commit, execute pre-commit run --all-files

[PranavBhatP]

Thanks for the PR @JATAYU000 . I've dropped some comments on the PR.

[PranavBhatP]

Conventionally for v2, all the layers of the models' architecture are present in the layers directory. Many of the layers you are using here can be directly imported from this directory, I see a lot of commonality. I would suggest only adding new layers (if not already present in layers) as a subdirectory - layers/_<layer-type>. sub_modules.py is a v1 convention

[JATAYU000]

I found some changes in some layers and let it be in submodules for now in draft pr, will fix this .

[PranavBhatP]

self.use_norm is not required in the model code since it will be handled by the D1/D2 layers. Normalization and denomalization need not be handled here. Simply return dec_out.

[JATAYU000]

Oh Thank you for pointing that out.

[PranavBhatP]

Can you add a few more test cases here?

[PranavBhatP]

Docstring for model hyperparameters is missing.

[JATAYU000]

I would suggest only adding new layers (if not already present in layers) as a subdirectory - layers/_

@PranavBhatP The EncoderLayer in layers/_encoders/ requires cross_attention , but iTransformer only needs self_attention.
Should I make cross_attention optional in the existing layer? or create a separate EncoderLayer for iTransformer?

[fkiraly]

re layers, I would do as follows:

• if exact same layer is available in layers, reuse it
• add new layers in layers
• if layer with modification is needed, add it as a separate layer
• optionally - this PR but also can be later PR - check if multiple similar layers can be "unified" in a single layer with more parameters

[PranavBhatP]

optionally - this PR but also can be later PR - check if multiple similar layers can be "unified" in a single layer with more parameters

@fkiraly seems like a nice good first issue?