FAQ

[Thrift-dev] [jira] [Updated] (THRIFT-563) Support for Multiplexing Services on any Transport, Protocol and Server

Walter Huf (JIRA)
Jan 15, 2013 at 4:54 am
[ https://issues.apache.org/jira/browse/THRIFT-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Walter Huf updated THRIFT-563:
------------------------------

Attachment: multiplex.py

Python port
Support for Multiplexing Services on any Transport, Protocol and Server
-----------------------------------------------------------------------

Key: THRIFT-563
URL: https://issues.apache.org/jira/browse/THRIFT-563
Project: Thrift
Issue Type: New Feature
Components: Java - Library
Reporter: Rob Slifka
Attachments: multiplex.py, THRIFT-563.patch


*Motivation and Benefits*
We plan to use Thrift with a large number of functions communicating among (at least) three languages. To keep maintainability high, we hope to avoid a single service defined with a large number of functions. This would require monolithic, unwieldy service implementations as the number of functions grows.
Breaking up our API into multiple IDLs gives us multiple abstract base classes, which provides more flexibility in the object-oriented design of our server platform.
Before our changes, the alternative was to open up additional ports with smaller service implementations on each. We'd rather not open additional ports.
*Our Approach*
We pursued an approach with the following in mind:
- No modifications to existing Thrift code.
- No modification to the Thrift protocol as described in the whitepaper.
- No modification to any {{TServer}}, {{TProtocol}} or {{TTransport}}.
- No need for any new {{TServer}} implementation. Works with any {{TServer}} implementation.
- Work with any combination of {{TServer}}, {{TProtocol}} or {{TTransport}}.
- Avoid language-specific features, to ease implementation in other languages.
*Additions to Thrift*
Convenience class:
- {{TProtocolDecorator}} (extends {{TProtocol}}). This is a no-op decorator around the {{TProtocol}} abstract class.
For use by clients:
- {{TMultiplexedProtocol}} (extends {{TProtocolDecorator}}). This decorates any {{TProtocol}} by modifying the behaviour of {{writeMessageBegin(TMessage)}} to change {{TMessage.name}} from {{function_name}} to {{service_name + separator + function_name}}.
For use by the server:
- {{TMultiplexedProcessor}} (implements {{TProcessor}}). It should be used to communicate with a client that was written using {{TMultiplexedProtocol}}. It removes {{service_name + separator}} from {{TMessage.name}}, turning it back into the standard message format. It then brokers the service request to the {{TProcessor}} which is registered to handle requests for that service.
*Sample Usage - Client*
In this example, we've chosen to use {{TBinaryProtocol}} with two services: {{Calculator}} and {{WeatherReport}}.
{code}
TSocket transport = new TSocket("localhost", 9090);
transport.open();
TBinaryProtocol protocol = new TBinaryProtocol(transport);
TMultiplexedProtocol mp = new TMultiplexedProtocol(protocol, "Calculator");
Calculator.Client service = new Calculator.Client(mp);
TMultiplexedProtocol mp2 = new TMultiplexedProtocol(protocol, "WeatherReport");
WeatherReport.Client service2 = new WeatherReport.Client(mp2);
System.out.println(service.add(2,2));
System.out.println(service2.getTemperature());
{code}
*Sample Usage - Server*
{code}
TMultiplexedProcessor processor = new TMultiplexedProcessor();
processor.registerProcessor(
"Calculator",
new Calculator.Processor(new CalculatorHandler()));
processor.registerProcessor(
"WeatherReport",
new WeatherReport.Processor(new WeatherReportHandler()));
TServerTransport t = new TServerSocket(9090);
TSimpleServer server = new TSimpleServer(processor, t);
server.serve();
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
reply

Search Discussions

3 responses

  • Walter Huf (JIRA) at Jan 15, 2013 at 4:56 am
    [ https://issues.apache.org/jira/browse/THRIFT-563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13553487#comment-13553487 ]

    Walter Huf commented on THRIFT-563:
    -----------------------------------

    I have ported the given patch into Python, and tested it with a simple server and client.
    On the server side, you make your normal processors, and then construct a TMultiplexedProcessor and run .registerProcessor to add your normal processors.
    On the client side, you create your normal protocol, and then create several TMultiplexedProtocols, passing in your main protocol and a service name. Then you create the service's client, passing the corresponding TMultiplexedProtocol.
    Support for Multiplexing Services on any Transport, Protocol and Server
    -----------------------------------------------------------------------

    Key: THRIFT-563
    URL: https://issues.apache.org/jira/browse/THRIFT-563
    Project: Thrift
    Issue Type: New Feature
    Components: Java - Library
    Reporter: Rob Slifka
    Attachments: multiplex.py, THRIFT-563.patch


    *Motivation and Benefits*
    We plan to use Thrift with a large number of functions communicating among (at least) three languages. To keep maintainability high, we hope to avoid a single service defined with a large number of functions. This would require monolithic, unwieldy service implementations as the number of functions grows.
    Breaking up our API into multiple IDLs gives us multiple abstract base classes, which provides more flexibility in the object-oriented design of our server platform.
    Before our changes, the alternative was to open up additional ports with smaller service implementations on each. We'd rather not open additional ports.
    *Our Approach*
    We pursued an approach with the following in mind:
    - No modifications to existing Thrift code.
    - No modification to the Thrift protocol as described in the whitepaper.
    - No modification to any {{TServer}}, {{TProtocol}} or {{TTransport}}.
    - No need for any new {{TServer}} implementation. Works with any {{TServer}} implementation.
    - Work with any combination of {{TServer}}, {{TProtocol}} or {{TTransport}}.
    - Avoid language-specific features, to ease implementation in other languages.
    *Additions to Thrift*
    Convenience class:
    - {{TProtocolDecorator}} (extends {{TProtocol}}). This is a no-op decorator around the {{TProtocol}} abstract class.
    For use by clients:
    - {{TMultiplexedProtocol}} (extends {{TProtocolDecorator}}). This decorates any {{TProtocol}} by modifying the behaviour of {{writeMessageBegin(TMessage)}} to change {{TMessage.name}} from {{function_name}} to {{service_name + separator + function_name}}.
    For use by the server:
    - {{TMultiplexedProcessor}} (implements {{TProcessor}}). It should be used to communicate with a client that was written using {{TMultiplexedProtocol}}. It removes {{service_name + separator}} from {{TMessage.name}}, turning it back into the standard message format. It then brokers the service request to the {{TProcessor}} which is registered to handle requests for that service.
    *Sample Usage - Client*
    In this example, we've chosen to use {{TBinaryProtocol}} with two services: {{Calculator}} and {{WeatherReport}}.
    {code}
    TSocket transport = new TSocket("localhost", 9090);
    transport.open();
    TBinaryProtocol protocol = new TBinaryProtocol(transport);
    TMultiplexedProtocol mp = new TMultiplexedProtocol(protocol, "Calculator");
    Calculator.Client service = new Calculator.Client(mp);
    TMultiplexedProtocol mp2 = new TMultiplexedProtocol(protocol, "WeatherReport");
    WeatherReport.Client service2 = new WeatherReport.Client(mp2);
    System.out.println(service.add(2,2));
    System.out.println(service2.getTemperature());
    {code}
    *Sample Usage - Server*
    {code}
    TMultiplexedProcessor processor = new TMultiplexedProcessor();
    processor.registerProcessor(
    "Calculator",
    new Calculator.Processor(new CalculatorHandler()));
    processor.registerProcessor(
    "WeatherReport",
    new WeatherReport.Processor(new WeatherReportHandler()));
    TServerTransport t = new TServerSocket(9090);
    TSimpleServer server = new TSimpleServer(processor, t);
    server.serve();
    {code}
    --
    This message is automatically generated by JIRA.
    If you think it was sent incorrectly, please contact your JIRA administrators
    For more information on JIRA, see: http://www.atlassian.com/software/jira
  • Walter Huf (JIRA) at Jan 15, 2013 at 4:56 am
    [ https://issues.apache.org/jira/browse/THRIFT-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Walter Huf updated THRIFT-563:
    ------------------------------

    Comment: was deleted

    (was: Python port)
    Support for Multiplexing Services on any Transport, Protocol and Server
    -----------------------------------------------------------------------

    Key: THRIFT-563
    URL: https://issues.apache.org/jira/browse/THRIFT-563
    Project: Thrift
    Issue Type: New Feature
    Components: Java - Library
    Reporter: Rob Slifka
    Attachments: multiplex.py, THRIFT-563.patch


    *Motivation and Benefits*
    We plan to use Thrift with a large number of functions communicating among (at least) three languages. To keep maintainability high, we hope to avoid a single service defined with a large number of functions. This would require monolithic, unwieldy service implementations as the number of functions grows.
    Breaking up our API into multiple IDLs gives us multiple abstract base classes, which provides more flexibility in the object-oriented design of our server platform.
    Before our changes, the alternative was to open up additional ports with smaller service implementations on each. We'd rather not open additional ports.
    *Our Approach*
    We pursued an approach with the following in mind:
    - No modifications to existing Thrift code.
    - No modification to the Thrift protocol as described in the whitepaper.
    - No modification to any {{TServer}}, {{TProtocol}} or {{TTransport}}.
    - No need for any new {{TServer}} implementation. Works with any {{TServer}} implementation.
    - Work with any combination of {{TServer}}, {{TProtocol}} or {{TTransport}}.
    - Avoid language-specific features, to ease implementation in other languages.
    *Additions to Thrift*
    Convenience class:
    - {{TProtocolDecorator}} (extends {{TProtocol}}). This is a no-op decorator around the {{TProtocol}} abstract class.
    For use by clients:
    - {{TMultiplexedProtocol}} (extends {{TProtocolDecorator}}). This decorates any {{TProtocol}} by modifying the behaviour of {{writeMessageBegin(TMessage)}} to change {{TMessage.name}} from {{function_name}} to {{service_name + separator + function_name}}.
    For use by the server:
    - {{TMultiplexedProcessor}} (implements {{TProcessor}}). It should be used to communicate with a client that was written using {{TMultiplexedProtocol}}. It removes {{service_name + separator}} from {{TMessage.name}}, turning it back into the standard message format. It then brokers the service request to the {{TProcessor}} which is registered to handle requests for that service.
    *Sample Usage - Client*
    In this example, we've chosen to use {{TBinaryProtocol}} with two services: {{Calculator}} and {{WeatherReport}}.
    {code}
    TSocket transport = new TSocket("localhost", 9090);
    transport.open();
    TBinaryProtocol protocol = new TBinaryProtocol(transport);
    TMultiplexedProtocol mp = new TMultiplexedProtocol(protocol, "Calculator");
    Calculator.Client service = new Calculator.Client(mp);
    TMultiplexedProtocol mp2 = new TMultiplexedProtocol(protocol, "WeatherReport");
    WeatherReport.Client service2 = new WeatherReport.Client(mp2);
    System.out.println(service.add(2,2));
    System.out.println(service2.getTemperature());
    {code}
    *Sample Usage - Server*
    {code}
    TMultiplexedProcessor processor = new TMultiplexedProcessor();
    processor.registerProcessor(
    "Calculator",
    new Calculator.Processor(new CalculatorHandler()));
    processor.registerProcessor(
    "WeatherReport",
    new WeatherReport.Processor(new WeatherReportHandler()));
    TServerTransport t = new TServerSocket(9090);
    TSimpleServer server = new TSimpleServer(processor, t);
    server.serve();
    {code}
    --
    This message is automatically generated by JIRA.
    If you think it was sent incorrectly, please contact your JIRA administrators
    For more information on JIRA, see: http://www.atlassian.com/software/jira
  • Patrik Lindblom (JIRA) at Jan 15, 2013 at 9:46 am
    [ https://issues.apache.org/jira/browse/THRIFT-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Patrik Lindblom updated THRIFT-563:
    -----------------------------------

    Attachment: multiplex_c++.tar

    Added C++ implementation of the classes.
    Support for Multiplexing Services on any Transport, Protocol and Server
    -----------------------------------------------------------------------

    Key: THRIFT-563
    URL: https://issues.apache.org/jira/browse/THRIFT-563
    Project: Thrift
    Issue Type: New Feature
    Components: Java - Library
    Reporter: Rob Slifka
    Attachments: multiplex_c++.tar, multiplex.py, THRIFT-563.patch


    *Motivation and Benefits*
    We plan to use Thrift with a large number of functions communicating among (at least) three languages. To keep maintainability high, we hope to avoid a single service defined with a large number of functions. This would require monolithic, unwieldy service implementations as the number of functions grows.
    Breaking up our API into multiple IDLs gives us multiple abstract base classes, which provides more flexibility in the object-oriented design of our server platform.
    Before our changes, the alternative was to open up additional ports with smaller service implementations on each. We'd rather not open additional ports.
    *Our Approach*
    We pursued an approach with the following in mind:
    - No modifications to existing Thrift code.
    - No modification to the Thrift protocol as described in the whitepaper.
    - No modification to any {{TServer}}, {{TProtocol}} or {{TTransport}}.
    - No need for any new {{TServer}} implementation. Works with any {{TServer}} implementation.
    - Work with any combination of {{TServer}}, {{TProtocol}} or {{TTransport}}.
    - Avoid language-specific features, to ease implementation in other languages.
    *Additions to Thrift*
    Convenience class:
    - {{TProtocolDecorator}} (extends {{TProtocol}}). This is a no-op decorator around the {{TProtocol}} abstract class.
    For use by clients:
    - {{TMultiplexedProtocol}} (extends {{TProtocolDecorator}}). This decorates any {{TProtocol}} by modifying the behaviour of {{writeMessageBegin(TMessage)}} to change {{TMessage.name}} from {{function_name}} to {{service_name + separator + function_name}}.
    For use by the server:
    - {{TMultiplexedProcessor}} (implements {{TProcessor}}). It should be used to communicate with a client that was written using {{TMultiplexedProtocol}}. It removes {{service_name + separator}} from {{TMessage.name}}, turning it back into the standard message format. It then brokers the service request to the {{TProcessor}} which is registered to handle requests for that service.
    *Sample Usage - Client*
    In this example, we've chosen to use {{TBinaryProtocol}} with two services: {{Calculator}} and {{WeatherReport}}.
    {code}
    TSocket transport = new TSocket("localhost", 9090);
    transport.open();
    TBinaryProtocol protocol = new TBinaryProtocol(transport);
    TMultiplexedProtocol mp = new TMultiplexedProtocol(protocol, "Calculator");
    Calculator.Client service = new Calculator.Client(mp);
    TMultiplexedProtocol mp2 = new TMultiplexedProtocol(protocol, "WeatherReport");
    WeatherReport.Client service2 = new WeatherReport.Client(mp2);
    System.out.println(service.add(2,2));
    System.out.println(service2.getTemperature());
    {code}
    *Sample Usage - Server*
    {code}
    TMultiplexedProcessor processor = new TMultiplexedProcessor();
    processor.registerProcessor(
    "Calculator",
    new Calculator.Processor(new CalculatorHandler()));
    processor.registerProcessor(
    "WeatherReport",
    new WeatherReport.Processor(new WeatherReportHandler()));
    TServerTransport t = new TServerSocket(9090);
    TSimpleServer server = new TSimpleServer(processor, t);
    server.serve();
    {code}
    --
    This message is automatically generated by JIRA.
    If you think it was sent incorrectly, please contact your JIRA administrators
    For more information on JIRA, see: http://www.atlassian.com/software/jira

Related Discussions

Discussion Navigation
viewthread | post

1 user in discussion

Patrik Lindblom (JIRA): 4 posts