Skip to content

Commit 8d5c958

Browse files
authored
Update 2024-08-07-flexattention.md
1 parent 60b9999 commit 8d5c958

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

_posts/2024-08-07-flexattention.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
layout: blog_detail
33
title: "FlexAttention: The Flexibility of PyTorch with the Performance of FlashAttention"
4-
author: "Team PyTorch: Horace He, Driss Guessous, Yanbo Liang, Joy Dong"
4+
author: "Team PyTorch: Driss Guessous, Yanbo Liang, Joy Dong, Horace He"
55
---
66

77
![a cartoon chart flexing his muscles](/assets/images/flexattention/fg1.jpg){:style="width:100%"}
@@ -131,7 +131,7 @@ Alibi is similar to relative positional encodings with one exception \- it has a
131131
alibi_bias = generate_alibi_bias() # [num_heads]
132132

133133
def alibi(score, b, h, q_idx, kv_idx):
134-
bias = alibi_bias[h] * (q_idx - kv_idx)
134+
bias = alibi_bias[h] * (kv_idx - q_idx)
135135
return score + bias
136136
```
137137

0 commit comments

Comments
 (0)