Its expected behavior that converted leads remain in your leads dashboard and will show a 'Closed' status. This is to help track how well the app is performing.
You can filter these out before messaging the whole group by doing the following: Click Lead Segmentation.
Toggle on 'Filter by conversion status' > click Apply.
Click Save segment for easy future access to this list / easy messaging.