SEOUL, South Korea - North Korea canceled a high-level meeting Wednesday with South Korea and also threatened to call off a historic summit planned for next month with the United States due to ongoing military exercises between the South and the U.S., South Korea's Yonhap news agency reported.

The two Koreas were set to hold a meeting Wednesday at a border village to discuss setting up military and Red Cross talks to reduce border tension and restart reunions between families separated by the Korean War.

Yonhap says North Korea's Korean Central News Agency (KCNA) reported that Wednesday's meeting was canceled and that Pyongyang was questioning whether next month's summit between North Korea's Kim Jong Un and President Trump can also take place as planned.

In response, State Department spokeswoman Heather Nauert said that the U.S. "will continue to plan the meeting between President Trump and Kim Jong Un."

"The United States will also have to undertake careful deliberations about the fate of the planned North Korea-U.S. summit in light of this provocative military ruckus jointly conducted with the South Korean authorities," KCNA reported.

The Pentagon says the military exercise is a routine, annual event that is purely defensive in nature. Army Col. Rob Manning says Exercise Max Thunder 2018 is designed to improve the abilities of the U.S. and South Korea to operate together. It began Monday and is slated to run through May 25, and is expected to include aircraft from the U.S. Air Force, Army, Navy and Marine Corps, as it has in the past.

Last year it included as many as 1,200 U.S. personnel and about 640 South Koreans as well as various aircraft including F-16 fighter jets, F-18 Hornets and EA-18G Growlers from the Navy's electronic attack squadron. Manning says the defensive nature of these combined exercises has been clear for many decades and has not changed.

In March, then-CIA Director and current Secretary of State Mike Pompeo told "Face the Nation" that Kim has allowed the U.S. to "continue our exercises on the peninsula something that's been fought over for decades." He also made similar remarks the same day on Fox News, saying Kim has "got to continue to allow us to perform our militarily necessary exercises on the peninsula and then he's got to make sure that he leaves on the table that discussion for denuclearization."

In March, South Korean National Security Adviser Chung Eui-Yong said Kim "understands that the routine joint military exercises between the Republic of Korea and the United States must continue" after meeting with Mr. Trump at the White House.

CBS News' Kylie Atwood and Steven Portnoy contributed to this report.

© 2018 CBS Interactive Inc. All Rights Reserved. This material may not be published, broadcast, rewritten, or redistributed. The Associated Press contributed to this report.