No reason was given for Li's removal and no replacement was announced.
Li had not appeared in public since August 29, when he gave a speech at a Chinese-African peace forum.
He is the second senior leader to be ousted after Foreign Minister Qin Gang was abruptly removed from office without explanation in July.
According to state broadcaster CCTV, both were also stripped of positions as state councillors.
Li's removal means China is without a defense minister as it prepares to host foreign defense ministers at the Beijing Xiangshan Forum, which takes place next week.
Li reportedly investigated for corruption
Li, 65, had only been in the job since March.
The Reuters news agency reported last month that he was being investigated for suspected corruption related to equipment procurement and development.
Chinese politics experts say President Xi Jinping handpicked both Li and Qin.
It makes their removal less than a year on the job particularly notable and raises questions about the stability of the leadership team around Xi, reports DW.