I would check the paper, but I think they fine tune a pre trained local LM. They also created their own encodings to account for the structure of tabular data, such as the column headers, entity rows etc.
I will note though, from what I remember the table sizes were pretty small.
master-leaf t1_je5hhu6 wrote
Reply to comment by jaxolingo in [D] The best way to train an LLM on company data by jaxolingo
I would check the paper, but I think they fine tune a pre trained local LM. They also created their own encodings to account for the structure of tabular data, such as the column headers, entity rows etc.
I will note though, from what I remember the table sizes were pretty small.