MPC Submission of Observations¶

This tutorial provides information on how to submit an ADES-formated file of observations to the Minor Planet Center.¶

This tutorial assumes you have successfully created and validated an ADES-formated file of observations.

Information on how to undertake the precursor step of constructing an ADES-formated file of observations can be found at:
- to be inserted
Information and documentation on the ADES standard can be found at:
- https://www.minorplanetcenter.net/mpcops/documentation/ades/
- https://github.com/IAU-ADES/ADES-Master

In the examples below we use python) code to demonstrate the submission of observations, but one can use other tools such as cURL to submit observations.

Further information and documentation concerning the submission of data to the MPC can be found at:

Import Packages¶

Here we import the standard python packages we use in this tutorial

In [ ]:

Copied!





import requests
import tempfile
import atexit
import os
import requests
import tempfile
import atexit
import os

Sample ADES Files¶

Here we download sample XML and PSV files from the MPC's website, and save them to local (temporary) files.

These files are known to be valid.

If you have created your own ADES file(s) for submission to the MPC, then this download of sample files is not necessary.

The local, temporary files will be deleted after this notebook exits.

In [ ]:

Copied!





# Define the files that we want fetch & save 
mpc_urls = [
    "https://www.minorplanetcenter.net/media/ades/goodsubmit.xml.txt", 
    "https://www.minorplanetcenter.net/media/ades/goodsubmit.psv.txt"]

local_filenames = ["sample.xml", "sample.psv"]
temp_filepaths = {} 

# Get the data and save it
for local_filename, mpc_url in zip(local_filenames, mpc_urls):
    with tempfile.NamedTemporaryFile(mode='w', suffix=local_filename[-4:], delete=False) as f:  # Open temp file
        f.write(requests.get(mpc_url).text)  # Write the data to temp file 
        temp_filepaths[local_filename[-3:]] = f.name  # Save the filepath of the temp file 
        print(f"Saved {mpc_url} to: {temp_filepaths[local_filename[-3:]]}")
    
        # Register a cleanup function that will execute when the notebook exits. 
        atexit.register(lambda: os.unlink(f.name) if os.path.exists(f.name) else None)
# Define the files that we want fetch & save 
mpc_urls = [
    "https://www.minorplanetcenter.net/media/ades/goodsubmit.xml.txt", 
    "https://www.minorplanetcenter.net/media/ades/goodsubmit.psv.txt"]

local_filenames = ["sample.xml", "sample.psv"]
temp_filepaths = {} 

# Get the data and save it
for local_filename, mpc_url in zip(local_filenames, mpc_urls):
    with tempfile.NamedTemporaryFile(mode='w', suffix=local_filename[-4:], delete=False) as f:  # Open temp file
        f.write(requests.get(mpc_url).text)  # Write the data to temp file 
        temp_filepaths[local_filename[-3:]] = f.name  # Save the filepath of the temp file 
        print(f"Saved {mpc_url} to: {temp_filepaths[local_filename[-3:]]}")
    
        # Register a cleanup function that will execute when the notebook exits. 
        atexit.register(lambda: os.unlink(f.name) if os.path.exists(f.name) else None)

Test Submission¶

Here we demonstrate the submission of the above sample ADES files to the MPC's test submission URLs.

We use the test endpoint as we do not want to spam the MPC's main pipeline with unnecessary submissions.

If you wish to test the submission of your own ADES file, please replace the filepath(s) with your own file(s).

XML Test Submission: Successful¶

Here we demonstrate sending our sample XML file (saved in temp_filepaths['xml']) to the test-XML end-point, https://minorplanetcenter.net/submit_xml_test.

We see that the submission was successful and that a Submission-ID was generated and returned.

In [ ]:

Copied!





# URL of the test end-point to which we will submit 
url = "https://minorplanetcenter.net/submit_xml_test"  

# Meta data to be communicated 
data = {
    'ack': 'MPC submission tutorial',  # Acknowledgment message (required):
    'ac2': 'my@email.adr',  # Address to which subsequent communications/notifications should be sent (required)
    'obj_type': 'NEO'  # Object Type Flag (optional): This example indicaties the observations are believed to be of a known NEO 
}

# Read the XML file, send to url, collect response
with open(temp_filepaths['xml'], 'rb') as f:
    response = requests.post(url, data=data, files={'source': f.read()})

# Print some of the details of the response
print(response.status_code)
print(response.text)
# URL of the test end-point to which we will submit 
url = "https://minorplanetcenter.net/submit_xml_test"  

# Meta data to be communicated 
data = {
    'ack': 'MPC submission tutorial',  # Acknowledgment message (required):
    'ac2': 'my@email.adr',  # Address to which subsequent communications/notifications should be sent (required)
    'obj_type': 'NEO'  # Object Type Flag (optional): This example indicaties the observations are believed to be of a known NEO 
}

# Read the XML file, send to url, collect response
with open(temp_filepaths['xml'], 'rb') as f:
    response = requests.post(url, data=data, files={'source': f.read()})

# Print some of the details of the response
print(response.status_code)
print(response.text)

PSV Test Submission: Successful¶

Here we demonstrate sending our sample PSV file (saved in temp_filepaths['psv']) to the test-PSV end-point, https://minorplanetcenter.net/submit_psv_test.

We see that the submission was successful and that a Submission-ID was generated and returned

In [ ]:

Copied!





# URL of the test end-point to which we will submit 
url = "https://minorplanetcenter.net/submit_psv_test"  

# Meta data to be communicated 
data = {
    'ack': 'MPC submission tutorial',  # Acknowledgment message (required):
    'ac2': 'my@email.adr',  # Address to which subsequent communications/notifications should be sent (required)
    'obj_type': 'NEO'  # Object Type Flag (optional): This example indicaties the observations are believed to be of a known NEO 
}

# Read the XML file, send to url, collect response
with open(temp_filepaths['psv'], 'rb') as f:
    response = requests.post(url, data=data, files={'source': f.read()})

# Print some of the details of the response
print(response.status_code)
print(response.text)
# URL of the test end-point to which we will submit 
url = "https://minorplanetcenter.net/submit_psv_test"  

# Meta data to be communicated 
data = {
    'ack': 'MPC submission tutorial',  # Acknowledgment message (required):
    'ac2': 'my@email.adr',  # Address to which subsequent communications/notifications should be sent (required)
    'obj_type': 'NEO'  # Object Type Flag (optional): This example indicaties the observations are believed to be of a known NEO 
}

# Read the XML file, send to url, collect response
with open(temp_filepaths['psv'], 'rb') as f:
    response = requests.post(url, data=data, files={'source': f.read()})

# Print some of the details of the response
print(response.status_code)
print(response.text)

XML Test Submission: Unsuccessful¶

Here we demonstrate sending our sample XML file (saved in temp_filepaths['xml']) to the test-XML end-point, https://minorplanetcenter.net/submit_xml_test.

Here we deliberately omit the required ack field from the data dictionary, resulting in the submission being unsuccessful, and thus no submission-ID is returned.

In [ ]:

Copied!





# URL of the test end-point to which we will submit 
url = "https://minorplanetcenter.net/submit_xml_test"  

# Meta data to be communicated 
# NB: The required `ack` field is omitted. 
data = {
    'ac2': 'my@email.adr',  # Address to which subsequent communications/notifications should be sent (required)
    'obj_type': 'NEO'  # Object Type Flag (optional): This example indicaties the observations are believed to be of a known NEO 
}

# Read the XML file, send to url, collect response
with open(temp_filepaths['xml'], 'rb') as f:
    response = requests.post(url, data=data, files={'source': f.read()})

# Print some of the details of the response
print(response.status_code)
print(response.text)
# URL of the test end-point to which we will submit 
url = "https://minorplanetcenter.net/submit_xml_test"  

# Meta data to be communicated 
# NB: The required `ack` field is omitted. 
data = {
    'ac2': 'my@email.adr',  # Address to which subsequent communications/notifications should be sent (required)
    'obj_type': 'NEO'  # Object Type Flag (optional): This example indicaties the observations are believed to be of a known NEO 
}

# Read the XML file, send to url, collect response
with open(temp_filepaths['xml'], 'rb') as f:
    response = requests.post(url, data=data, files={'source': f.read()})

# Print some of the details of the response
print(response.status_code)
print(response.text)

PSV Test Submission: Initial Success of Invalid Data¶

Here we demonstrate sending a string of data containing no meaningful observational data, and that does not pass ADES validation.

We see that, because the required meta-data is supplied, a submission-id is assigned and returned, and the data is passed into the system for subsequent validation.

This implies that the assignment of a submission-id does not mean that the ADES file passes validation, and hence it does not mean that any observations contained in such a submission will necessarily be accepted.

To understand whether a submission has been fully accepted and ingested, we recommend that submitters make use of the Submission Status API available at

https://data.minorplanetcenter.net/api/submission-status

and for which a tutorial is available at

https://docs.minorplanetcenter.net/tutorials/notebooks/mpc_tutorial_api_submission_status.html

In [ ]:

Copied!





# URL of the test end-point to which we will submit 
url = "https://minorplanetcenter.net/submit_psv_test"  

# Meta data to be communicated 
# NB: The required `ack` field is omitted. 
data = {
    'ack': 'MPC submission tutorial',  # Acknowledgment message (required):
    'ac2': 'my@email.adr',  # Address to which subsequent communications/notifications should be sent (required)
    'obj_type': 'NEO'  # Object Type Flag (optional): This example indicaties the observations are believed to be of a known NEO 
}

# Read the XML file, send to url, collect response
bad_psv_content = """# version=2017
permID |trkSub |mode|stn |obsTime                |ra         |dec        |rmsRA|rmsDec|rmsCorr|astCat|mag  |rmsMag|band|photCat|logSNR|notes|remarks"""
response = requests.post(url, data=data, files={'source': bad_psv_content})

# Print some of the details of the response
print(response.status_code)
print(response.text)
# URL of the test end-point to which we will submit 
url = "https://minorplanetcenter.net/submit_psv_test"  

# Meta data to be communicated 
# NB: The required `ack` field is omitted. 
data = {
    'ack': 'MPC submission tutorial',  # Acknowledgment message (required):
    'ac2': 'my@email.adr',  # Address to which subsequent communications/notifications should be sent (required)
    'obj_type': 'NEO'  # Object Type Flag (optional): This example indicaties the observations are believed to be of a known NEO 
}

# Read the XML file, send to url, collect response
bad_psv_content = """# version=2017
permID |trkSub |mode|stn |obsTime                |ra         |dec        |rmsRA|rmsDec|rmsCorr|astCat|mag  |rmsMag|band|photCat|logSNR|notes|remarks"""
response = requests.post(url, data=data, files={'source': bad_psv_content})

# Print some of the details of the response
print(response.status_code)
print(response.text)

XML Submission to Production Endpoint¶

Here we demonstrate sending an XML file (saved above into temp_filepaths['xml']) to the production XML end-point:

`https://minorplanetcenter.net/submit_xml`.

The only difference with respect to the test submissions above, is the change of the URL from https://minorplanetcenter.net/submit_xml_test to https://minorplanetcenter.net/submit_xml.

To reduce the risk of accidental, repeated submission of fake/test data into the MPC's production system , this tutorial deliberately converts the following cell to "markdown" rather then "code".

""" This cell deliberately set to markdown format (rather than code) to reduce the risk of accidental, repeated submission of fake/test data into the MPC's production system. """

URL of the test end-point to which we will submit¶

url = "https://minorplanetcenter.net/submit_xml"

Meta data to be communicated¶

data = { 'ack': 'MPC submission tutorial', # Acknowledgment message (required): 'ac2': 'my@email.adr', # Address to which subsequent communications/notifications should be sent (required) 'obj_type': 'NEO' # Object Type Flag (optional): This example indicaties the observations are believed to be of a known NEO }

Read the XML file, send to url, collect response¶

with open(temp_filepaths['xml'], 'rb') as f: response = requests.post(url, data=data, files={'source': f.read()})

Print some of the details of the response¶

print(response.status_code) print(response.text)

PSV Submission to Production Endpoint¶

Here we demonstrate sending an PSV file (saved above into temp_filepaths['psv']) to the production PSV end-point:

`https://minorplanetcenter.net/submit_psv`.

The only difference with respect to the test submissions above, is the change of the URL from https://minorplanetcenter.net/submit_psv_test to https://minorplanetcenter.net/submit_psv.

To reduce the risk of accidental, repeated submission of fake/test data into the MPC's production system , this tutorial deliberately converts the following cell to "markdown" rather then "code".

""" This cell deliberately set to markdown format (rather than code) to reduce the risk of accidental, repeated submission of fake/test data into the MPC's production system. """

URL of the test end-point to which we will submit¶

url = "https://minorplanetcenter.net/submit_psv"

Meta data to be communicated¶

data = { 'ack': 'MPC submission tutorial', # Acknowledgment message (required): 'ac2': 'my@email.adr', # Address to which subsequent communications/notifications should be sent (required) 'obj_type': 'NEO' # Object Type Flag (optional): This example indicaties the observations are believed to be of a known NEO }

Read the XML file, send to url, collect response¶

with open(temp_filepaths['psv'], 'rb') as f: response = requests.post(url, data=data, files={'source': f.read()})

Print some of the details of the response¶

print(response.status_code) print(response.text)