GitHub - celcilin/cellularflow: Memory-Augmented Continual Learning LLM architecture decoupling knowledge storage from sequence reasoning to eliminate catastrophic forgetting.
github.com
Memory-Augmented Continual Learning LLM architecture decoupling knowledge storage from sequence reasoning to eliminate catastrophic forgetting. - celcilin/cellularflow
  • Sims@lemmy.mlEnglish
    3·
    10 days ago

    Interesting, and impressive to just whip up a new learning architecture single handedly like this ! There was a paper on how to inject memory into the last layer(s) of a normal transformer architecture. Seemed a bit ‘hacky’ but not sure which is better. No matter, In a few months we’ll have several architectures and techniques to choose from, that all replace/augment primitive RAG systems, so the missing parts of the artificial brain is being developed everywhere.

    • ☆ Yσɠƚԋσʂ ☆@lemmy.mlOP
      2·
      10 days ago

      This really does seem like the next frontier. If a model can learn and adopt on the fly, it becomes far more useful. And I suspect that small models could become tuned to do specific tasks a lot better than a generic large model.