GGUF
The file format for a quantised model meant to be run locally, holding the weights and everything needed to load them in one file.
what it is made of · the vocabulary
Q4_K_Mquantised weights
If you download a model to run on your own machine you will almost certainly download a GGUF. The suffix — Q4_K_M and its relatives — names the precision and therefore the trade between size and quality.