About 20 trucks carrying food, water and medical supplies may be allowed to enter Gaza in the coming days.
Israel cut electricity, most water and stopped deliveries of food and medicine to Gaza following an attack by Hamas militants on 7 October.
Since then, Gaza's 2.1 million residents have been rapidly running out of basic supplies, reports BBC.
But leading humanitarian organisations warn the aid delivery will only be a drop in the ocean.
"The UN has reported that a minimum of 100 trucks of humanitarian assistance are needed in order to support the millions of civilians living in Gaza", Shaina Low of the Norwegian Refugee Council told the BBC.
Philippe Lazzarini, the commissioner-general of United Nations Relief and Works Agency (UNRWA), told the BBC that before the war about 500 trucks a day would enter Gaza with aid, fuel and other commodities.
That sentiment was echoed by Juliette Touma, the UN body's spokeswoman in Amman, who told the BBC that even before the war started 1.2 million people relied on food aid from UNRWA.
"Poverty is very, very high in the Gaza Strip. Already before the war the situation was desperate. Now it is becoming tragic," she said.
The agreement to deliver a limited amount of aid via Egypt's Rafah crossing was reached by US President Joe Biden and Egypt's President Abdul Fattah al-Sisi on Wednesday.
Mr Biden called Mr al-Sisi following a brief visit to Israel.
Israeli Prime Minister Benjamin Netanyahu confirmed on Wednesday that Israel would "not thwart" supplies going from Egypt to the civilian population in southern Gaza.
However, his government only agreed to allow food, water and medical supplies - not other much-needed supplies like fuel.
A UN report on Gaza said that fuel is a necessity, and a lack of fuel is contributing to the water crisis, as desalination plants and water pumps can no longer operate.
Mr Lazzarini said that if fuel could not be delivered, many more trucks will be needed to carry in water.