On the Shape tab, in the Remove group: Click Delete to delete the currently selected design element (Ctrl+D,D). You can also press the Delete key on the keyboard.
To delete all selected elements, press Ctrl+Shift+D.