Issues

• Exception for “ttft” in the run result

  flexflow/flexflow-serve#130 · Filter by author 2065037421 opened on Apr 28, 2025

• Error when using multiple SSMs: Assertion 'input_trees.size() == 1 && "currently using one ssm"' failed

  flexflow/flexflow-serve#120 · Filter by author bowgeem opened on Mar 27, 2025

• Build failure: undefined reference to `__fs_log_1'

  bug
  flexflow/flexflow-serve#88 · Filter by author suranap opened on Jan 7, 2025

• Hash function doesn't hash

  bug
  flexflow/flexflow-serve#89 · Filter by author suranap opened on Dec 14, 2024

• Unable to use ./inference/incr_decoding/incr_decoding on inference branch

  flexflow/flexflow-serve#1 · Filter by author hugolatendresse opened on Nov 26, 2024

• How to enable support for larger sequence lengths?

  flexflow/flexflow-serve#2 · Filter by author QAZWSX0827 opened on Nov 22, 2024
  3

• Performance issue when batch_size is 32

  flexflow/flexflow-serve#3 · Filter by author letheantest opened on Oct 22, 2024

• Stuck During Tree-Based Speculative Decoding with OPT Model

  flexflow/flexflow-serve#4 · Filter by author SeungjaeLim opened on Oct 16, 2024

• cuIpcGetMemHandle triggered CUDA out of memory when I use flexflow on one gpu

  flexflow/flexflow-serve#75 · Filter by author spacecat2002 opened on Sep 11, 2024
  4

• Error when I use larger batch size for spec-infer

  flexflow/flexflow-serve#6 · Filter by author lhr-30 opened on Sep 6, 2024
  1

• Issue with C++ inference for model meta-llama/Llama-2-70b-hf

  flexflow/flexflow-serve#5 · Filter by author DDDDDYTS opened on Jul 24, 2024

• Request for Graph Pruning Algorithm Code Location in FlexLLM

  flexflow/flexflow-serve#7 · Filter by author zbtrs opened on Jun 26, 2024

• Performance Issue

  enhancement
  flexflow/flexflow-serve#15 · Filter by author lethean287 opened on Apr 20, 2024
  1

  

• Flexflow attempts to register duplicate sharding functors

  bug
  flexflow/flexflow-serve#17 · Filter by author rohany opened on Apr 20, 2024
  2

  

• Operators support for updated speculative inference design

  flexflow/flexflow-serve#8 · Filter by author chenzhuofu opened on Apr 20, 2024
  6

  

• Simplify BatchConfig

  flexflow/flexflow-serve#9 · Filter by author zikun-li opened on Apr 12, 2024
  11

  

• Update on request manager

  flexflow/flexflow-serve#14 · Filter by author zwang86 opened on Apr 9, 2024
  4

• Can we modify the "expansion configuration" parameters, such as "token tree width" and "token tree depth"?

  question
  flexflow/flexflow-serve#53 · Filter by author hky011011 opened on Mar 27, 2024
  

• Error while importing import flexflow.serve as ff

  flexflow/flexflow-serve#18 · Filter by author goelayu opened on Mar 26, 2024
  4

  

• Modify BeamSearchBatchConfig to Support General Tree Structure.

  flexflow/flexflow-serve#13 · Filter by author zikun-li opened on Mar 15, 2024
  9

  

• FFModel 'null' in compile_inference

  flexflow/flexflow-serve#78 · Filter by author april-yyt opened on Mar 15, 2024
  

• Modify model cache path

  flexflow/flexflow-serve#19 · Filter by author lockshaw opened on Mar 12, 2024
  1

  

• Huggingface Format Convert & Upload PR Issues

  flexflow/flexflow-serve#20 · Filter by author april-yyt opened on Mar 1, 2024

• use the cuda graph for specinfer and it's blocked

  bug
  flexflow/flexflow-serve#46 · Filter by author lambda7xx opened on Feb 27, 2024
  2

• Does FlexFlow support P40?

  flexflow/flexflow-serve#45 · Filter by author wozwdaqian opened on Feb 26, 2024