p-morais
p-morais t1_izvyzit wrote
Reply to comment by patient_zer00 in [D] - Has Open AI said what ChatGPT's architecture is? What technique is it using to "remember" previous prompts? by 029187
It’s instructGPT, which is based on GPT3.5 with RLHF. People have reversed engineered that it uses a context window of 8,192 tokens and primed with a special prompt.
p-morais t1_iyr0wzl wrote
Reply to comment by Conscious_Heron_9133 in [D] PyTorch 2.0 Announcement by joshadel
All I want for Christmas is a strongly typed, low level, non-garbage-collected, safe programming language with pythonic syntax and first class tensor types and native compile-time autodiff. Is that too much to ask for?
p-morais t1_izvz91t wrote
Reply to comment by Duckdog2022 in [D] - Has Open AI said what ChatGPT's architecture is? What technique is it using to "remember" previous prompts? by 029187
Not “pretty unlikely”. The architecture is literally in the name: Generative Pretrained Transformer