Optimize the kernel implementation of layernorm with openmp #20895
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Xreki
reviewed
Oct 30, 2019
| rest_mask & 0x80 ? 0xffffffff : 0, rest_mask & 0x40 ? 0xffffffff : 0, | ||
| rest_mask & 0x20 ? 0xffffffff : 0, rest_mask & 0x10 ? 0xffffffff : 0, | ||
| rest_mask & 0x8 ? 0xffffffff : 0, rest_mask & 0x4 ? 0xffffffff : 0, | ||
| rest_mask & 0x2 ? 0xffffffff : 0, rest_mask & 0x1 ? 0xffffffff : 0); |
There was a problem hiding this comment.
可以拆成#pragma omp parallel和#paragma omp for两个部分，这样__m256i mask_vec就只需设置一次？
test=develop
test=develop
3 tasks
|
Ernie性能数据在：PaddlePaddle/benchmark#180 (comment) @Xreki 可以再看一下 |
luotao1
approved these changes
Oct 31, 2019
seiriosPlus
pushed a commit
to seiriosPlus/Paddle
that referenced
this pull request
Dec 9, 2019
seiriosPlus
pushed a commit
to seiriosPlus/Paddle
that referenced
this pull request
Dec 9, 2019
GaoWei8
pushed a commit
to GaoWei8/Paddle
that referenced
this pull request
Jan 9, 2020
Xreki
pushed a commit
that referenced
this pull request
Jan 10, 2020
…22198) * Optimize the kernel implementation of layernorm with openmp (#20895) * Add ernie c++ inference test (#21015) * Add ernie unit test test=develop * Add ernie unit test test=develop * Add ernie unit test test=develop * remove ngraph * optimize gpu test test=develop * optimize codes test=develop * fix cmake fails on inference_download_and_uncompress (#21185) * solve cmake fails on inference_download_and_uncompress test=develop * solve cmake fails on inference_download_and_uncompress test=develop * Add fc padding to improve mkl GEMM's performance when N and K are multiple of 128. (#20972) * Add fc padding to solve mkl performance test=develop * fix gpu pass and error information test=develop * fix fc_fuse_pass_test test=develop * fix error information test=develop * fix error information test=develop * fix name and add fc op padding test test=develop * fix attributes test=develop * optimize fc padding test=develop * fix test test=develop * Polish the codes of fc when needs padding (#21378) test=develop * Add ernie large c++ inference test (#21365) * add ernie-large test test=develop * add ernie large c++ inference test test=develop * Modify padding strategy: remove weight copy in fc padding (#21650) test=develop * optimize fc jit (#21878) test=develop Co-authored-by: Yihua Xu <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Based on investigation of ERNIE, we found that layernorm is lack of multi-threads JIT implemention. This PR is to add the layernorm multi-threads optimizing by using OpenMP.
But based on initial benchmark with ERNIE, single thread layernorm performance seems to be worse than before.
CPU Model: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz